00000080:00010000:2.0:1788571635.965442:0:75453:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571635.965445:0:75453:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3ce2b 00000080:00010000:2.0:1788571635.965448:0:75453:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3ce2b from it ffffab0ec1cbfcd0 00010000:00010000:2.0:1788571635.965450:0:75453:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51758200/0xef30c340bfb3ce2b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f5a4d expref: -99 pid: 75446 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:2.0:1788571635.965457:0:75453:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51758200/0xef30c340bfb3ce2b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5a4d expref: -99 pid: 75446 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00000080:00200000:2.0:1788571635.965465:0:75453:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:2.0:1788571635.965470:0:75453:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571635.965476:0:75453:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571635.965482:0:75453:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51758200/0xef30c340bfb3ce2b lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5a4d expref: -99 pid: 75446 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:2.0:1788571635.965489:0:75453:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51758200/0xef30c340bfb3ce2b lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5a4d expref: -99 pid: 75446 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00000080:00200000:2.0:1788571635.965495:0:75453:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571635.965498:0:75453:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571635.965500:0:75453:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3ce2b 00000080:00010000:2.0:1788571635.965503:0:75453:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3ce2b from it ffffab0ec1cbfdb0 00010000:00010000:2.0:1788571635.965506:0:75453:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51758200/0xef30c340bfb3ce2b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f5a4d expref: -99 pid: 75446 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:2.0:1788571635.965529:0:75453:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51758200/0xef30c340bfb3ce2b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5a4d expref: -99 pid: 75446 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00000080:00200000:2.0:1788571635.965538:0:75453:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:2.0:1788571635.965544:0:75453:0:(namei.c:2062:ll_create_nd()) VFS Op:name=fsa-oleg213-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), flags=33188, excl=1 00000080:00200000:2.0:1788571635.965548:0:75453:0:(namei.c:2017:ll_mknod()) VFS Op:name=fsa-oleg213-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9ada58125b88) mode 100644 dev 0 00010000:00010000:2.0:1788571635.965576:0:75453:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51758200/0xef30c340bfb3ce2b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b82f5a4d expref: -99 pid: 75446 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00010000:00010000:2.0:1788571635.965636:0:75453:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51758200/0xef30c340bfb3ce2b lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f5a4d expref: -99 pid: 75446 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00010000:00010000:2.0:1788571635.965643:0:75453:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1788571635.965646:0:75453:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51758200/0xef30c340bfb3ce2b lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f5a4d expref: -99 pid: 75446 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00000020:00100000:2.0:1788571635.965674:0:75453:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1788571635.965690:0:75453:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada60a34000 x1875449613517952/t0(0) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788571635.965716:0:75453:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a34000 pname:cluuid:pid:xid:nid:opc:job mcreate:35eb2d60-2de0-4b00-9815-d5153edd0057:75453:1875449613517952:192.168.202.113@tcp:36:mcreate.0 00000100:00100000:2.0:1788571635.965808:0:75453:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada509eb0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788571635.989210:0:75453:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a34000 pname:cluuid:pid:xid:nid:opc:job mcreate:35eb2d60-2de0-4b00-9815-d5153edd0057:75453:1875449613517952:192.168.202.113@tcp:36:mcreate.0 00000080:00200000:2.0:1788571635.989262:0:75453:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x20000fe02:0x20:0x0](ffff9ada58196c08) 00000080:00200000:2.0:1788571635.989305:0:75453:0:(namei.c:2070:ll_create_nd()) VFS Op:name=fsa-oleg213-client.virtnet, unhashed 1 00000100:00100000:2.0:1788571636.038101:0:75456:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f5f80 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:75456:1875449613518080:192.168.202.113@tcp:101:rm.0 00010000:00010000:2.0:1788571636.038133:0:75456:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f800/0xef30c340bfb3ce32 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5a69 expref: -99 pid: 75456 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571636.038147:0:75456:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f800/0xef30c340bfb3ce32 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5a69 expref: -99 pid: 75456 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571636.038166:0:75456:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f800/0xef30c340bfb3ce32 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f5a69 expref: -99 pid: 75456 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571636.038178:0:75456:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f800/0xef30c340bfb3ce32 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5a69 expref: -99 pid: 75456 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571636.038193:0:75456:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f800/0xef30c340bfb3ce32 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f5a69 expref: -99 pid: 75456 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571636.038202:0:75456:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f800/0xef30c340bfb3ce32 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f5a69 expref: -99 pid: 75456 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571636.038209:0:75456:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f800/0xef30c340bfb3ce32 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f5a69 expref: -99 pid: 75456 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1788571636.038221:0:75456:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada603f5f80 x1875449613518080/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788571652 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1788571636.038236:0:75456:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f800/0xef30c340bfb3ce32 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5a69 expref: -99 pid: 75456 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571636.038249:0:75456:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:2.0:1788571636.038256:0:75456:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:2.0:1788571636.038285:0:75456:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:2.0:1788571636.038291:0:75456:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3ce32 00000080:00010000:2.0:1788571636.038296:0:75456:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571636.038299:0:75456:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3ce32 from it ffffab0ec185bb58 00010000:00010000:2.0:1788571636.038303:0:75456:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f800/0xef30c340bfb3ce32 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f5a69 expref: -99 pid: 75456 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571636.038312:0:75456:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f800/0xef30c340bfb3ce32 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5a69 expref: -99 pid: 75456 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571636.038339:0:75456:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:2.0:1788571636.038361:0:75456:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=fsa-oleg213-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), flags=8 00000080:00200000:2.0:1788571636.038365:0:75456:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=fsa-oleg213-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=getattr 00000002:00010000:2.0:1788571636.038388:0:75456:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: fsa-oleg213-client.virtnet,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 01 00010000:00010000:2.0:1788571636.038428:0:75456:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8c00/0xef30c340bfb3ce39 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 75456 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571636.038435:0:75456:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8c00/0xef30c340bfb3ce39 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 75456 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571636.038443:0:75456:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8c00/0xef30c340bfb3ce39 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 75456 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571636.038462:0:75456:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37480 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:75456:1875449613518208:192.168.202.113@tcp:101:rm.0 00000100:00100000:2.0:1788571636.038564:0:75456:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada509eb0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788571636.044214:0:75456:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37480 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:75456:1875449613518208:192.168.202.113@tcp:101:rm.0 00010000:00010000:2.0:1788571636.044246:0:75456:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8c00/0xef30c340bfb3ce39 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5a85 expref: -99 pid: 75456 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571636.044269:0:75456:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8c00/0xef30c340bfb3ce39 lrc: 4/1,0 mode: --/PR res: [0x20000fe02:0x20:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5a85 expref: -99 pid: 75456 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571636.044283:0:75456:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8c00/0xef30c340bfb3ce39 lrc: 4/1,0 mode: --/PR res: [0x20000fe02:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5a85 expref: -99 pid: 75456 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571636.044294:0:75456:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8c00/0xef30c340bfb3ce39 lrc: 4/1,0 mode: PR/PR res: [0x20000fe02:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f5a85 expref: -99 pid: 75456 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571636.044306:0:75456:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8c00/0xef30c340bfb3ce39 lrc: 4/1,0 mode: PR/PR res: [0x20000fe02:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5a85 expref: -99 pid: 75456 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571636.044319:0:75456:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8c00/0xef30c340bfb3ce39 lrc: 5/2,0 mode: PR/PR res: [0x20000fe02:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f5a85 expref: -99 pid: 75456 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571636.044327:0:75456:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8c00/0xef30c340bfb3ce39 lrc: 5/2,0 mode: PR/PR res: [0x20000fe02:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f5a85 expref: -99 pid: 75456 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571636.044335:0:75456:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8c00/0xef30c340bfb3ce39 lrc: 4/1,0 mode: PR/PR res: [0x20000fe02:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f5a85 expref: -99 pid: 75456 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1788571636.044348:0:75456:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada60a37480 x1875449613518208/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 600/584 e 0 to 0 dl 1788571652 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1788571636.044361:0:75456:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8c00/0xef30c340bfb3ce39 lrc: 3/1,0 mode: PR/PR res: [0x20000fe02:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5a85 expref: -99 pid: 75456 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571636.044374:0:75456:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589999618/32 (0 0) 00000080:00200000:2.0:1788571636.044381:0:75456:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1788571636.044401:0:75456:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x20000fe02:0x20:0x0], updating i_size 0 00000080:00200000:2.0:1788571636.044404:0:75456:0:(namei.c:147:ll_iget()) got inode: [0x20000fe02:0x20:0x0](ffff9ada58196c08): rc = 0 00000080:00200000:2.0:1788571636.044421:0:75456:0:(file.c:6843:ll_layout_conf()) [0x20000fe02:0x20:0x0]: layout version change: 4294967294 -> 4294967295 00000080:00010000:2.0:1788571636.044427:0:75456:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20000fe02:0x20:0x0](ffff9ada58196c08) for lock 0xef30c340bfb3ce39 00000080:00010000:2.0:1788571636.044438:0:75456:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20000fe02:0x20:0x0](ffff9ada58196c08) 00000080:00010000:2.0:1788571636.044442:0:75456:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3ce39 from it ffffab0ec185bb20 00010000:00010000:2.0:1788571636.044445:0:75456:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8c00/0xef30c340bfb3ce39 lrc: 3/1,0 mode: PR/PR res: [0x20000fe02:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f5a85 expref: -99 pid: 75456 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571636.044455:0:75456:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8c00/0xef30c340bfb3ce39 lrc: 2/0,0 mode: PR/PR res: [0x20000fe02:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f5a85 expref: -99 pid: 75456 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:2.0:1788571636.044500:0:75456:0:(file.c:6296:ll_getattr_dentry()) START file fsa-oleg213-client.virtnet:[0x20000fe02:0x20:0x0](ffff9ada58196c08), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1788571636.044507:0:75456:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x20000fe02:0x20:0x0](ffff9ada58196c08),name=fsa-oleg213-client.virtnet 00000002:00010000:2.0:1788571636.044568:0:75456:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x20000fe02:0x20:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571636.044578:0:75456:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8c00/0xef30c340bfb3ce39 lrc: 2/1,0 mode: PR/PR res: [0x20000fe02:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f5a85 expref: -99 pid: 75456 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571636.044588:0:75456:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8c00/0xef30c340bfb3ce39 lrc: 2/1,0 mode: PR/PR res: [0x20000fe02:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f5a85 expref: -99 pid: 75456 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571636.044596:0:75456:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571636.044600:0:75456:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x20000fe02:0x20:0x0](ffff9ada58196c08) 00000080:00010000:2.0:1788571636.044603:0:75456:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x20000fe02:0x20:0x0](ffff9ada58196c08) for lock 0xef30c340bfb3ce39 00000080:00010000:2.0:1788571636.044606:0:75456:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3ce39 from it ffffab0ec185bcb8 00010000:00010000:2.0:1788571636.044609:0:75456:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8c00/0xef30c340bfb3ce39 lrc: 3/1,0 mode: PR/PR res: [0x20000fe02:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f5a85 expref: -99 pid: 75456 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571636.044616:0:75456:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8c00/0xef30c340bfb3ce39 lrc: 2/0,0 mode: PR/PR res: [0x20000fe02:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f5a85 expref: -99 pid: 75456 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571636.044694:0:75456:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571636.044703:0:75456:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571636.044711:0:75456:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f800/0xef30c340bfb3ce32 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5a69 expref: -99 pid: 75456 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571636.044719:0:75456:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f800/0xef30c340bfb3ce32 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5a69 expref: -99 pid: 75456 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571636.044725:0:75456:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571636.044728:0:75456:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571636.044731:0:75456:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3ce32 00000080:00010000:2.0:1788571636.044734:0:75456:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3ce32 from it ffffab0ec185bc10 00010000:00010000:2.0:1788571636.044738:0:75456:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f800/0xef30c340bfb3ce32 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f5a69 expref: -99 pid: 75456 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571636.044744:0:75456:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f800/0xef30c340bfb3ce32 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5a69 expref: -99 pid: 75456 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571636.044752:0:75456:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:2.0:1788571636.044762:0:75456:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=fsa-oleg213-client.virtnet, flags=0 00000080:00200000:2.0:1788571636.044769:0:75456:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571636.044775:0:75456:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571636.044781:0:75456:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f800/0xef30c340bfb3ce32 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5a69 expref: -99 pid: 75456 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571636.044789:0:75456:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f800/0xef30c340bfb3ce32 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5a69 expref: -99 pid: 75456 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571636.044796:0:75456:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571636.044799:0:75456:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571636.044802:0:75456:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3ce32 00000080:00010000:2.0:1788571636.044806:0:75456:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3ce32 from it ffffab0ec185bd68 00010000:00010000:2.0:1788571636.044809:0:75456:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f800/0xef30c340bfb3ce32 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f5a69 expref: -99 pid: 75456 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571636.044816:0:75456:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f800/0xef30c340bfb3ce32 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5a69 expref: -99 pid: 75456 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1788571636.044824:0:75456:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:2.0:1788571636.044829:0:75456:0:(namei.c:2351:ll_unlink()) VFS Op:name=fsa-oleg213-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9ada58125b88) 00010000:00010000:2.0:1788571636.044879:0:75456:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f800/0xef30c340bfb3ce32 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b82f5a69 expref: -99 pid: 75456 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1788571636.044905:0:75456:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8c00/0xef30c340bfb3ce39 lrc: 2/0,0 mode: PR/PR res: [0x20000fe02:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0xdb7edb52b82f5a85 expref: -99 pid: 75456 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571636.044915:0:75456:0:(vvp_object.c:117:vvp_conf_set()) [0x20000fe02:0x20:0x0]: losing layout lock 00010000:00010000:2.0:1788571636.044955:0:75456:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8c00/0xef30c340bfb3ce39 lrc: 1/0,0 mode: --/PR res: [0x20000fe02:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0xdb7edb52b82f5a85 expref: -99 pid: 75456 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571636.044962:0:75456:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f800/0xef30c340bfb3ce32 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f5a69 expref: -99 pid: 75456 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1788571636.044968:0:75456:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:2.0:1788571636.044970:0:75456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8c00/0xef30c340bfb3ce39 lrc: 0/0,0 mode: --/PR res: [0x20000fe02:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0xdb7edb52b82f5a85 expref: -99 pid: 75456 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571636.044989:0:75456:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f800/0xef30c340bfb3ce32 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f5a69 expref: -99 pid: 75456 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:2.0:1788571636.045010:0:75456:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1788571636.045018:0:75456:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada60a37480 x1875449613518464/t0(0) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 512/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788571636.045266:0:75456:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37480 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:75456:1875449613518464:192.168.202.113@tcp:36:rm.0 00000100:00100000:2.0:1788571636.045353:0:75456:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada509eb0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788571636.050763:0:75456:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37480 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:75456:1875449613518464:192.168.202.113@tcp:36:rm.0 00000080:00200000:2.0:1788571636.050831:0:75456:0:(vvp_io.c:1826:vvp_io_init()) [0x20000fe02:0x20:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:2.0:1788571636.050843:0:75456:0:(vvp_io.c:304:vvp_io_fini()) [0x20000fe02:0x20:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1788571636.050851:0:75456:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20000fe02:0x20:0x0](ffff9ada58196c08) 00000100:00100000:2.0:1788571636.107407:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a49180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449613518592:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571641.208277:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613518848:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571641.208411:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613519104:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571641.208450:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:2.0:1788571641.208456:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a36680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449613518720:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571641.208483:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a36a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613518976:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571641.208502:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a36300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613519232:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571641.227683:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a36680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449613518720:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571641.227728:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a36a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613518976:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571641.227743:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a36300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613519232:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571641.227759:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613518848:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571651.470323:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a48380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613521024:192.168.202.113@tcp:400:kworker.0 00000100:00080000:2.0:1788571651.470352:0:2385:0:(recover.c:205:ptlrpc_request_handle_notconn()) import lustre-MDT0000-mdc-ffff9ada58b0a000 of lustre-MDT0000_UUID@192.168.202.113@tcp abruptly disconnected: reconnecting 00000100:00080000:2.0:1788571651.470360:0:2385:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:2.0:1788571651.470364:0:2385:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff9ada58b0a000: Connection to lustre-MDT0000 (at 192.168.202.113@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:2.0:1788571651.470372:0:2385:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788571651.470377:0:2385:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 3523 00000100:00080000:2.0:1788571651.470395:0:2385:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788571651.470410:0:2385:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788571651.470463:0:2385:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60a37800 x1875449613521152/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788571651.470500:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613520640:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571651.470519:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613520896:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571651.470556:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a49c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613520768:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571656.578480:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613521664:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571656.578505:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a48380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613521792:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571661.697356:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a49f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613522304:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571661.697388:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a49500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613522176:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571661.697403:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613522432:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571666.828162:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613522944:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571666.828190:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a48a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613522816:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571666.828206:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613523072:192.168.202.113@tcp:400:kworker.0 00000100:00000400:2.0:1788571667.641718:0:2385:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788571651/real 1788571651] req@ffff9ada60a49880 x1875449613520512/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 1 dl 1788571667 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788571667.729918:0:2385:0:(client.c:2548:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9ada60a49880 x1875449613520512/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 1 dl 1788571667 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788571667.729939:0:2385:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from FULL to DISCONN 00000100:02020000:2.0:1788571667.729943:0:2385:0:(import.c:186:ptlrpc_set_import_discon()) MGC192.168.202.113@tcp: Connection to MGS (at 192.168.202.113@tcp) was lost; in progress operations using this service will fail 00000100:00080000:2.0:1788571667.751193:0:2385:0:(import.c:192:ptlrpc_set_import_discon()) import MGS@192.168.202.113@tcp for MGC192.168.202.113@tcp not replayable, auto-deactivating 00000100:00080000:2.0:1788571667.751201:0:2385:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import MGS INVALID 00000100:00100000:2.0:1788571667.751213:0:2385:0:(client.c:3591:ptlrpc_abort_inflight()) @@@ inflight req@ffff9ada60a49880 x1875449613520512/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 1 dl 1788571667 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788571667.751489:0:2385:0:(client.c:3591:ptlrpc_abort_inflight()) @@@ inflight req@ffff9ada60a4aa00 x1875449613521280/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788571672 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788571667.751510:0:2385:0:(client.c:3591:ptlrpc_abort_inflight()) @@@ inflight req@ffff9ada60a4ad80 x1875449613521920/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788571677 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788571667.751520:0:2385:0:(client.c:3591:ptlrpc_abort_inflight()) @@@ inflight req@ffff9ada60a4a300 x1875449613522560/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788571682 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:2.0:1788571667.751531:0:2385:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808001 00000100:00080000:2.0:1788571667.751535:0:2385:0:(pinger.c:193:ptlrpc_pinger_ir_down()) IR down 10000000:01000000:2.0:1788571667.751537:0:2385:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808002 00000100:00080000:2.0:1788571667.751540:0:2385:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00100000:2.0:1788571667.751736:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449613522560:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571667.751747:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449613521920:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571667.751755:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449613521280:192.168.202.113@tcp:400:kworker.0 00000100:00080000:2.0:1788571672.952180:0:48662:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788571672.952193:0:48662:0:(pinger.c:247:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1788571672.952196:0:48662:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788571672.952200:0:48662:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788571672.952206:0:48662:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788571672.952220:0:48662:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 3585 00000100:00080000:2.0:1788571672.952233:0:48662:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788571672.952242:0:48662:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788571672.952258:0:48662:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788571672.952306:0:48662:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60a36300 x1875449613523840/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788571672.952582:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a36680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613524096:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571672.952623:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1788571672.952630:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a36a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613523968:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571672.952653:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a34e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613524224:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571672.985575:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a36680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613524096:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571672.985624:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a36a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613523968:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571672.985638:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a34e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613524224:192.168.202.113@tcp:400:kworker.0 00000100:00080000:2.0:1788571677.099116:0:75745:0:(import.c:1642:ptlrpc_invalidate_import_thread()) thread invalidate import MGC192.168.202.113@tcp to MGS@192.168.202.113@tcp 00000100:00100000:2.0:1788571677.099121:0:75745:0:(import.c:323:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 10000000:01000000:2.0:1788571677.099126:0:75745:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808003 00010000:00010000:2.0:1788571677.099143:0:75745:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.202.113@tcp lock: ffff9ada51616000/0xef30c340bfb3cdc2 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x110e400000000 nid: local remote: 0xdb7edb52b82f5641 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1788571677.099154:0:75745:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.202.113@tcp lock: ffff9ada51616000/0xef30c340bfb3cdc2 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x110f400000000 nid: local remote: 0xdb7edb52b82f5641 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1788571677.099161:0:75745:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1788571677.099181:0:75745:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571677.099193:0:75745:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.202.113@tcp lock: ffff9ada51616000/0xef30c340bfb3cdc2 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x590f400000000 nid: local remote: 0xdb7edb52b82f5641 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571677.099209:0:75745:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.202.113@tcp lock: ffff9ada5163f600/0xef30c340bfb3cdd0 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0xdb7edb52b82f566b expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1788571677.099339:0:75745:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.202.113@tcp lock: ffff9ada5163f600/0xef30c340bfb3cdd0 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0xdb7edb52b82f566b expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1788571677.099345:0:75745:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1788571677.099392:0:75745:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571677.099398:0:75745:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.202.113@tcp lock: ffff9ada5163f600/0xef30c340bfb3cdd0 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0xdb7edb52b82f566b expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571677.099419:0:75745:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.202.113@tcp lock: ffff9ada603e9e00/0xef30c340bfb3cdbb lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0xdb7edb52b82f563a expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1788571677.099426:0:75745:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.202.113@tcp lock: ffff9ada603e9e00/0xef30c340bfb3cdbb lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0xdb7edb52b82f563a expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1788571677.099432:0:75745:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:2.0:1788571677.099988:0:75745:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571677.099996:0:75745:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.202.113@tcp lock: ffff9ada603e9e00/0xef30c340bfb3cdbb lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0xdb7edb52b82f563a expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571677.100013:0:75745:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.202.113@tcp lock: ffff9ada5163ee00/0xef30c340bfb3cdc9 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0xdb7edb52b82f5664 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1788571677.100020:0:75745:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.202.113@tcp lock: ffff9ada5163ee00/0xef30c340bfb3cdc9 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0xdb7edb52b82f5664 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1788571677.100025:0:75745:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:2.0:1788571677.100173:0:75745:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571677.100179:0:75745:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.202.113@tcp lock: ffff9ada5163ee00/0xef30c340bfb3cdc9 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0xdb7edb52b82f5664 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1788571677.100197:0:75745:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from EVICTED to RECOVER 00000100:00080000:2.0:1788571677.100202:0:75745:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from RECOVER to FULL 10000000:01000000:2.0:1788571677.100204:0:75745:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808004 00000100:00080000:2.0:1788571677.100354:0:75745:0:(pinger.c:186:ptlrpc_pinger_ir_up()) IR up 00000100:02000000:2.0:1788571677.100361:0:75745:0:(import.c:1787:ptlrpc_import_recovery_state_machine()) MGC192.168.202.113@tcp: Connection restored to 192.168.202.113@tcp (at 192.168.202.113@tcp) 00000020:01000000:2.0:1788571679.124049:0:15207:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 52-51 (rc=0) 10000000:01000000:2.0:1788571679.124090:0:15207:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.202.113@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:2.0:1788571679.124108:0:15207:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.202.113@tcp lock: ffff9ada51616400/0xef30c340bfb3ce55 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b82f5cdf expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571679.124122:0:15207:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.202.113@tcp lock: ffff9ada51616400/0xef30c340bfb3ce55 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b82f5cdf expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1788571679.124134:0:15207:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:2.0:1788571679.124137:0:15207:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9ada58b0a000 from 7 10000000:01000000:2.0:1788571679.124141:0:15207:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:2.0:1788571679.124193:0:15207:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.202.113@tcp lock: ffff9ada603e9c00/0xef30c340bfb3ce5c lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571679.124203:0:15207:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.202.113@tcp lock: ffff9ada603e9c00/0xef30c340bfb3ce5c lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571679.124217:0:15207:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.202.113@tcp lock: ffff9ada603e9c00/0xef30c340bfb3ce5c lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571679.124251:0:15207:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a34e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449613525760:192.168.202.113@tcp:101:ll_cfg_requeue.0 00000100:00100000:2.0:1788571679.124349:0:15207:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada53736600 going to sleep for 11 seconds 00000020:01000000:2.0:1788571679.161963:0:15207:0:(obd_config.c:2145:class_config_parse_llog()) Processed log params gen 7-6 (rc=0) 10000000:01000000:2.0:1788571679.161990:0:15207:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.202.113@tcp: configuration from log 'params' succeeded (0). 00010000:00010000:2.0:1788571679.162005:0:15207:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.202.113@tcp lock: ffff9ada603e9c00/0xef30c340bfb3ce5c lrc: 3/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b82f5ced expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571679.162019:0:15207:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.202.113@tcp lock: ffff9ada603e9c00/0xef30c340bfb3ce5c lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b82f5ced expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1788571679.162030:0:15207:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:2.0:1788571679.162034:0:15207:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9ada42e899c0 from 1 10000000:01000000:2.0:1788571679.162038:0:15207:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:2.0:1788571679.162088:0:15207:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.202.113@tcp lock: ffff9ada603e8600/0xef30c340bfb3ce63 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571679.162098:0:15207:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.202.113@tcp lock: ffff9ada603e8600/0xef30c340bfb3ce63 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571679.162112:0:15207:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.202.113@tcp lock: ffff9ada603e8600/0xef30c340bfb3ce63 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571679.162146:0:15207:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a36a00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449613526144:192.168.202.113@tcp:101:ll_cfg_requeue.0 00000100:00100000:2.0:1788571679.162251:0:15207:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada53736b40 going to sleep for 11 seconds 00010000:00010000:2.0:1788571682.194115:0:33423:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 0 locks of 0 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1788571682.194139:0:33423:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 0 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1788571682.194148:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571682.194150:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571682.194151:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571683.704129:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788571684.220561:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449613527040:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571684.220592:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a48700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613527296:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571684.220619:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613527552:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571684.220634:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a48000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613527168:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571684.220645:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a49880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613527424:192.168.202.113@tcp:400:kworker.0 00010000:00010000:2.0:1788571686.775245:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571689.847080:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571692.922412:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571695.994151:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571699.063449:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:2.0:1788571700.531619:0:75426:0:(llite_lib.c:2427:ll_setattr_raw()) START file f31.replay-single-1:[0x20000fe02:0x1e:0x0](ffff9ada5819e3c8) current size 0, valid attrs 41, mode 8000, uid 1, gid 0, new size 18446744071748348645, atime 1788571700.000000000, mtime 1788571700.000000000, ctime 1788571700.000000000 00000020:00100000:2.0:1788571700.531725:0:75426:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1788571700.531740:0:75426:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada60a36680 x1875449613530880/t0(0) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 488/512 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788571700.531767:0:75426:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a36680 pname:cluuid:pid:xid:nid:opc:job multiop:35eb2d60-2de0-4b00-9815-d5153edd0057:75426:1875449613530880:192.168.202.113@tcp:36:multiop.0 00000100:00100000:2.0:1788571700.531961:0:75426:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada5118a600 going to sleep for 11 seconds 00000080:00200000:2.0:1788571700.572774:0:75426:0:(vvp_io.c:304:vvp_io_fini()) [0x20000fe02:0x1e:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200008:2.0:1788571700.572800:0:75426:0:(llite_lib.c:2686:ll_setattr_raw()) COMPLETED file f31.replay-single-1:[0x20000fe02:0x1e:0x0](ffff9ada5819e3c8) current size 0, valid attrs 10000041, mode 8000, uid 1, gid 0, new size 18446744071748348645, atime 1788571700.000000000, mtime 1788571700.000000000, ctime 1788571700.000000000, rc 0 00000080:00200008:2.0:1788571700.572834:0:75426:0:(file.c:6296:ll_getattr_dentry()) START file f31.replay-single-1:[0x20000fe02:0x1e:0x0](ffff9ada5819e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1788571700.572841:0:75426:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x20000fe02:0x1e:0x0](ffff9ada5819e3c8),name=f31.replay-single-1 00000002:00010000:2.0:1788571700.572897:0:75426:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x20000fe02:0x1e:0x0]) in obj [0x20000fe02:0x1e:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571700.572912:0:75426:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 30 res 8589999618/30 (0 0) 00010000:00010000:2.0:1788571700.572964:0:75426:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9600/0xef30c340bfb3ce78 lrc: 3/1,0 mode: --/CR res: [0x20000fe02:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 75426 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571700.572974:0:75426:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9600/0xef30c340bfb3ce78 lrc: 3/1,0 mode: --/CR res: [0x20000fe02:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 75426 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571700.572983:0:75426:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9600/0xef30c340bfb3ce78 lrc: 3/1,0 mode: --/CR res: [0x20000fe02:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 75426 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571700.573008:0:75426:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a36a00 pname:cluuid:pid:xid:nid:opc:job multiop:35eb2d60-2de0-4b00-9815-d5153edd0057:75426:1875449613531264:192.168.202.113@tcp:101:multiop.0 00000100:00100000:2.0:1788571700.573093:0:75426:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada5118a840 going to sleep for 11 seconds 00000100:00100000:2.0:1788571700.590693:0:75426:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f1c00 pname:cluuid:pid:xid:nid:opc:job multiop:35eb2d60-2de0-4b00-9815-d5153edd0057:75426:1875449613531520:192.168.202.113@tcp:35:multiop.0 00000002:00100000:2.0:1788571700.590732:0:75426:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9ada516f0380 x1875449613516160/t184683593733(184683593733) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/608 e 0 to 0 dl 1788571695 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1788571700.590777:0:75426:0:(vvp_io.c:1826:vvp_io_init()) [0x20000fe02:0x1e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1788571700.590791:0:75426:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1788571700.590818:0:75426:0:(vvp_io.c:304:vvp_io_fini()) [0x20000fe02:0x1e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1788571700.590825:0:75426:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x20000fe02:0x1e:0x0](ffff9ada5819e3c8) 00000008:00010000:2.0:1788571700.590875:0:75426:0:(osc_object.c:202:osc_object_ast_clear()) ### update lock size 0 blocks 0 [cma]time: 0 0 0 by oinfo size 0 blocks 0 [cma]time 1788571700 0 0 ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000b5cf9624/0xef30c340bfb3ce24 lrc: 1/0,0 mode: PR/PR res: [0x280000401:0x5a2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0xdb7edb52b82f5a2a expref: -99 pid: 75436 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571700.682411:0:75286:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571700.682441:0:75286:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571700.682461:0:75286:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f600/0xef30c340bfb3ce6a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5efa expref: -99 pid: 75848 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00010000:00010000:2.0:1788571700.682476:0:75286:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f600/0xef30c340bfb3ce6a lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5efa expref: -99 pid: 75848 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00000080:00200000:2.0:1788571700.682485:0:75286:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571700.682494:0:75286:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571700.682498:0:75286:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3ce6a 00000080:00010000:2.0:1788571700.682505:0:75286:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3ce6a from it ffffab0ec1463b58 00010000:00010000:2.0:1788571700.682510:0:75286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f600/0xef30c340bfb3ce6a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f5efa expref: -99 pid: 75848 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00010000:00010000:2.0:1788571700.682519:0:75286:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f600/0xef30c340bfb3ce6a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5efa expref: -99 pid: 75848 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 0 00000080:00200000:2.0:1788571700.682536:0:75286:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:2.0:1788571700.682559:0:75286:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f31.replay-single-1, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), flags=9 00000080:00200000:2.0:1788571700.682563:0:75286:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f31.replay-single-1, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=getattr 00000002:00010000:2.0:1788571700.682578:0:75286:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f31.replay-single-1,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 01 00010000:00010000:2.0:1788571700.682625:0:75286:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3ce8d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 75286 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571700.682633:0:75286:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3ce8d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 75286 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571700.682644:0:75286:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3ce8d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 75286 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571700.682686:0:75286:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a34e00 pname:cluuid:pid:xid:nid:opc:job bash:35eb2d60-2de0-4b00-9815-d5153edd0057:75286:1875449613532544:192.168.202.113@tcp:101:bash.0 00000100:00100000:2.0:1788571700.682789:0:75286:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada5118a780 going to sleep for 11 seconds 00010000:00010000:2.0:1788571702.135016:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571705.208404:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788571706.346901:0:33410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875449638436224 00000100:00100000:2.0:1788571706.346918:0:33410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875449638436224 00000100:00100000:2.0:1788571706.346977:0:33410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875449638436352 00000100:00100000:2.0:1788571706.346981:0:33410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875449638436352 00000100:00100000:2.0:1788571706.347008:0:33410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.113@tcp, seq: 217 00000100:00100000:2.0:1788571706.347020:0:33410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9ada60a4a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_002:LOV_OSC_UUID+4:17011:x1875449638436224:12345-192.168.202.113@tcp:104: 00010000:00010000:2.0:1788571706.347043:0:33410:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000b5cf9624/0xef30c340bfb3ce24 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x420000010000 nid: local remote: 0xdb7edb52b82f5a2a expref: -99 pid: 75436 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571706.347141:0:33410:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1788571706.347150:0:33410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9ada60a4a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_002:LOV_OSC_UUID+4:17011:x1875449638436224:12345-192.168.202.113@tcp:104: Request processed in 133us (545us total) trans 0 rc 0/0 00000100:00100000:2.0:1788571706.347167:0:33410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.113@tcp, seq: 217 00000100:00100000:2.0:1788571706.347186:0:33410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.113@tcp, seq: 218 00000100:00100000:2.0:1788571706.347193:0:33410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9ada60a48e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_002:LOV_OSC_UUID+4:43976:x1875449638436352:12345-192.168.202.113@tcp:104: 00010000:00010000:2.0:1788571706.347202:0:33410:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 00000000e6015be1/0xef30c340bfb3ce47 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x5a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x420000010000 nid: local remote: 0xdb7edb52b82f5ab6 expref: -99 pid: 75523 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571706.347234:0:33410:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to priority blp list, 2 blwis in pool 00000100:00100000:2.0:1788571706.347240:0:33410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9ada60a48e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_002:LOV_OSC_UUID+4:43976:x1875449638436352:12345-192.168.202.113@tcp:104: Request processed in 47us (575us total) trans 0 rc 0/0 00000100:00100000:2.0:1788571706.347246:0:33410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.113@tcp, seq: 218 00010000:00010000:2.0:1788571706.347356:0:33423:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1788571706.347364:0:33423:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000b5cf9624/0xef30c340bfb3ce24 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x420000010000 nid: local remote: 0xdb7edb52b82f5a2a expref: -99 pid: 75436 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571706.347374:0:33423:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada603e9200 already unused, calling callback (ffffffffc102e300) 00010000:00010000:2.0:1788571706.347381:0:33423:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000b5cf9624/0xef30c340bfb3ce24 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x428400010000 nid: local remote: 0xdb7edb52b82f5a2a expref: -99 pid: 75436 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571706.347434:0:33423:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000b5cf9624/0xef30c340bfb3ce24 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0xdb7edb52b82f5a2a expref: -99 pid: 75436 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571706.347442:0:33423:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1788571706.347475:0:33423:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000b5cf9624/0xef30c340bfb3ce24 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x5a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0xdb7edb52b82f5a2a expref: -99 pid: 75436 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571706.347490:0:33423:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000b5cf9624/0xef30c340bfb3ce24 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0xdb7edb52b82f5a2a expref: -99 pid: 75436 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571706.347513:0:33423:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1788571706.347517:0:33423:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 00000000e6015be1/0xef30c340bfb3ce47 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x5a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x420000010000 nid: local remote: 0xdb7edb52b82f5ab6 expref: -99 pid: 75523 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571706.347525:0:33423:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada5163f200 already unused, calling callback (ffffffffc102e300) 00010000:00010000:2.0:1788571706.347529:0:33423:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 00000000e6015be1/0xef30c340bfb3ce47 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x5a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x428400010000 nid: local remote: 0xdb7edb52b82f5ab6 expref: -99 pid: 75523 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571706.347548:0:33423:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 00000000e6015be1/0xef30c340bfb3ce47 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x5a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0xdb7edb52b82f5ab6 expref: -99 pid: 75523 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571706.347555:0:33423:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1788571706.347561:0:33423:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 00000000e6015be1/0xef30c340bfb3ce47 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x5a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0xdb7edb52b82f5ab6 expref: -99 pid: 75523 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571706.347569:0:33423:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 00000000e6015be1/0xef30c340bfb3ce47 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x5a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0xdb7edb52b82f5ab6 expref: -99 pid: 75523 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571706.347584:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571706.347586:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571706.347587:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788571706.347607:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613533440:192.168.202.113@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1788571706.347675:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1788571706.347682:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a34000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613533568:192.168.202.113@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788571708.285176:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571711.352267:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:2.0:1788571712.482475:0:76963:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1788571712.482493:0:76963:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571712.482529:0:76963:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571712.482556:0:76963:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51758400/0xef30c340bfb3cea2 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5fa9 expref: -99 pid: 76915 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571712.482576:0:76963:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51758400/0xef30c340bfb3cea2 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5fa9 expref: -99 pid: 76915 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571712.482586:0:76963:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571712.482596:0:76963:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571712.482601:0:76963:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3cea2 00000080:00010000:2.0:1788571712.482611:0:76963:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cea2 from it ffffab0ec0d1fcb8 00010000:00010000:2.0:1788571712.482616:0:76963:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51758400/0xef30c340bfb3cea2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f5fa9 expref: -99 pid: 76915 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571712.482628:0:76963:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51758400/0xef30c340bfb3cea2 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5fa9 expref: -99 pid: 76915 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200008:2.0:1788571712.483325:0:76963:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 0, flags 0, foreign 0 00000080:00200000:2.0:1788571712.483331:0:76963:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571712.483346:0:76963:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571712.483357:0:76963:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51758400/0xef30c340bfb3cea2 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5fa9 expref: -99 pid: 76915 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1788571712.483370:0:76963:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51758400/0xef30c340bfb3cea2 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5fa9 expref: -99 pid: 76915 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1788571712.483376:0:76963:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571712.483381:0:76963:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571712.483384:0:76963:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3cea2 00000080:00010000:2.0:1788571712.483389:0:76963:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cea2 from it ffffab0ec0d1fcb8 00010000:00010000:2.0:1788571712.483392:0:76963:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51758400/0xef30c340bfb3cea2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f5fa9 expref: -99 pid: 76915 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1788571712.483401:0:76963:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51758400/0xef30c340bfb3cea2 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5fa9 expref: -99 pid: 76915 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:2.0:1788571712.483531:0:76963:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571712.483541:0:76963:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571712.483548:0:76963:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51758400/0xef30c340bfb3cea2 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5fa9 expref: -99 pid: 76915 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:2.0:1788571712.483558:0:76963:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51758400/0xef30c340bfb3cea2 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5fa9 expref: -99 pid: 76915 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:2.0:1788571712.483564:0:76963:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571712.483568:0:76963:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571712.483571:0:76963:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3cea2 00000080:00010000:2.0:1788571712.483575:0:76963:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cea2 from it ffffab0ec0d1fbb0 00010000:00010000:2.0:1788571712.483577:0:76963:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51758400/0xef30c340bfb3cea2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f5fa9 expref: -99 pid: 76915 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:2.0:1788571712.483586:0:76963:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51758400/0xef30c340bfb3cea2 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5fa9 expref: -99 pid: 76915 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00000080:00200000:2.0:1788571712.483595:0:76963:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:2.0:1788571712.483604:0:76963:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200080:2.0:1788571712.483721:0:76963:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdff3f2150 00800000:00000080:2.0:1788571712.483740:0:76963:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f000 uarg=0000000000000000 00800000:00000004:2.0:1788571712.483749:0:76963:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: age 3618, max_age 3629 00000100:00100000:2.0:1788571712.483808:0:76963:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a36300 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:76963:1875449613534720:192.168.202.113@tcp:41:lfs.0 00000100:00100000:2.0:1788571712.483943:0:76963:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada47dd2300 going to sleep for 11 seconds 00000100:00080000:2.0:1788571712.507430:0:2382:0:(import.c:1176:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to target with instance 44 00000100:00080000:2.0:1788571712.507487:0:2382:0:(import.c:1382:ptlrpc_connect_interpret()) @@@ lustre-MDT0000-mdc-ffff9ada58b0a000: evicting (reconnect/recover flags not set: 4) req@ffff9ada60a4a680 x1875449613534848/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/416 e 0 to 0 dl 1788571722 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788571712.507504:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to EVICTED 00000100:02020000:2.0:1788571712.507512:0:2382:0:(import.c:1707:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff9ada58b0a000: This client was evicted by lustre-MDT0000; in progress operations using this service will fail. 00000100:00080000:2.0:1788571712.537077:0:2382:0:(import.c:1718:ptlrpc_import_recovery_state_machine()) evicted from lustre-MDT0000_UUID@192.168.202.113@tcp; invalidating 00000100:00100000:2.0:1788571712.543701:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449613534848:192.168.202.113@tcp:38:lfs.0 00010000:00010000:2.0:1788571714.424975:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:2.0:1788571714.479233:0:77018:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1788571714.479255:0:77018:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571714.479289:0:77018:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571714.479373:0:77018:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3cea9 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 77018 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571714.479388:0:77018:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3cea9 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77018 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571714.479399:0:77018:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3cea9 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77018 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571714.479441:0:77018:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37800 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:77018:1875449613535360:192.168.202.113@tcp:101:lfs.0 00000100:00100000:2.0:1788571714.479719:0:77018:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada45229c00 going to sleep for 11 seconds 00000100:00100000:2.0:1788571714.488439:0:77018:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37800 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:77018:1875449613535360:192.168.202.113@tcp:101:lfs.0 00010000:00010000:2.0:1788571714.488482:0:77018:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3cea9 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5fbe expref: -99 pid: 77018 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571714.488498:0:77018:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3cea9 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5fbe expref: -99 pid: 77018 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571714.488581:0:77018:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3cea9 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f5fbe expref: -99 pid: 77018 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571714.488597:0:77018:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3cea9 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5fbe expref: -99 pid: 77018 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571714.488614:0:77018:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3cea9 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f5fbe expref: -99 pid: 77018 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571714.488623:0:77018:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3cea9 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f5fbe expref: -99 pid: 77018 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571714.488637:0:77018:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3cea9 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f5fbe expref: -99 pid: 77018 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1788571714.488651:0:77018:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada60a37800 x1875449613535360/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788571730 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1788571714.489112:0:77018:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3cea9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5fbe expref: -99 pid: 77018 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571714.489133:0:77018:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:2.0:1788571714.489142:0:77018:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:2.0:1788571714.489178:0:77018:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:2.0:1788571714.489184:0:77018:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3cea9 00000080:00010000:2.0:1788571714.489190:0:77018:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571714.489194:0:77018:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cea9 from it ffffab0ec0d7fcb8 00010000:00010000:2.0:1788571714.489199:0:77018:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3cea9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f5fbe expref: -99 pid: 77018 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571714.489211:0:77018:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3cea9 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5fbe expref: -99 pid: 77018 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:2.0:1788571714.490988:0:77018:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 0, flags 0, foreign 0 00000080:00200000:2.0:1788571714.490999:0:77018:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571714.491094:0:77018:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571714.491118:0:77018:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3cea9 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5fbe expref: -99 pid: 77018 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571714.491141:0:77018:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3cea9 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5fbe expref: -99 pid: 77018 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571714.491149:0:77018:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571714.491159:0:77018:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571714.491163:0:77018:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3cea9 00000080:00010000:2.0:1788571714.491171:0:77018:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cea9 from it ffffab0ec0d7fcb8 00010000:00010000:2.0:1788571714.491175:0:77018:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3cea9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f5fbe expref: -99 pid: 77018 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571714.491184:0:77018:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3cea9 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5fbe expref: -99 pid: 77018 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571714.491407:0:77018:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571714.491416:0:77018:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571714.491436:0:77018:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3cea9 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5fbe expref: -99 pid: 77018 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571714.491446:0:77018:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3cea9 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5fbe expref: -99 pid: 77018 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571714.491452:0:77018:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571714.491457:0:77018:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571714.491460:0:77018:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3cea9 00000080:00010000:2.0:1788571714.491464:0:77018:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cea9 from it ffffab0ec0d7fbb0 00010000:00010000:2.0:1788571714.491467:0:77018:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3cea9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f5fbe expref: -99 pid: 77018 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571714.491476:0:77018:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3cea9 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5fbe expref: -99 pid: 77018 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1788571714.491485:0:77018:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:2.0:1788571714.491494:0:77018:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200080:2.0:1788571714.492413:0:77018:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec7b4fa50 00800000:00000080:2.0:1788571714.492449:0:77018:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f800 uarg=0000000000000000 00800000:00000004:2.0:1788571714.492460:0:77018:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: age 3618, max_age 3631 00000100:00100000:2.0:1788571714.492587:0:77018:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37800 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:77018:1875449613535488:192.168.202.113@tcp:41:lfs.0 00000100:00100000:2.0:1788571714.493409:0:77018:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada45229c00 going to sleep for 11 seconds 00000080:00200008:2.0:1788571714.546872:0:76910:0:(file.c:479:ll_file_release()) START file f32.replay-single:[0x20000fe02:0x21:0x0](ffff9ada5819db88), flags 100002 00010000:00010000:2.0:1788571717.500674:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:2.0:1788571722.455660:0:77618:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571722.455703:0:77618:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571722.455728:0:77618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3cea9 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5fbe expref: -99 pid: 77018 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1788571722.455747:0:77618:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3cea9 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5fbe expref: -99 pid: 77018 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1788571722.455756:0:77618:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571722.455766:0:77618:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571722.455770:0:77618:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3cea9 00000080:00010000:2.0:1788571722.455780:0:77618:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cea9 from it ffffab0ec0d9bbf8 00010000:00010000:2.0:1788571722.455784:0:77618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3cea9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f5fbe expref: -99 pid: 77018 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1788571722.455801:0:77618:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3cea9 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5fbe expref: -99 pid: 77018 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:2.0:1788571722.455821:0:77618:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:2.0:1788571722.455839:0:77618:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571722.455846:0:77618:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571722.455874:0:77618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3cea9 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5fbe expref: -99 pid: 77018 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:2.0:1788571722.455884:0:77618:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3cea9 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5fbe expref: -99 pid: 77018 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:2.0:1788571722.455891:0:77618:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571722.455895:0:77618:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571722.455898:0:77618:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3cea9 00000080:00010000:2.0:1788571722.455902:0:77618:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cea9 from it ffffab0ec0d9bb38 00010000:00010000:2.0:1788571722.455905:0:77618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3cea9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f5fbe expref: -99 pid: 77018 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:2.0:1788571722.455914:0:77618:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3cea9 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f5fbe expref: -99 pid: 77018 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00000080:00200000:2.0:1788571722.455922:0:77618:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:2.0:1788571722.455928:0:77618:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33a.replay-single-0, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), file ffff9ada43222c00, open_flags 8042, mode 81a4 opened 0 00000080:00200000:2.0:1788571722.455938:0:77618:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33a.replay-single-0, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000100:00100000:2.0:1788571722.456013:0:77618:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35f80 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:77618:1875449613537024:192.168.202.113@tcp:700:createmany.0 00000100:00100000:2.0:1788571722.456267:0:77618:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada50f7a0c0 going to sleep for 11 seconds 00000080:00200000:2.0:1788571722.637148:0:77618:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:2.0:1788571722.637158:0:77618:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3ced3 00000080:00010000:2.0:1788571722.637165:0:77618:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571722.637169:0:77618:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3ced3 from it ffffab0ec0d9bbf8 00010000:00010000:2.0:1788571722.637174:0:77618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51758400/0xef30c340bfb3ced3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6051 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.637186:0:77618:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51758400/0xef30c340bfb3ced3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6051 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571722.637231:0:77618:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:2.0:1788571722.637276:0:77618:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571722.637298:0:77618:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571722.637310:0:77618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51758400/0xef30c340bfb3ced3 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6051 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571722.637322:0:77618:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51758400/0xef30c340bfb3ced3 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6051 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571722.637329:0:77618:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571722.637333:0:77618:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571722.637337:0:77618:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3ced3 00000080:00010000:2.0:1788571722.637340:0:77618:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3ced3 from it ffffab0ec0d9bb38 00010000:00010000:2.0:1788571722.637344:0:77618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51758400/0xef30c340bfb3ced3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6051 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571722.637351:0:77618:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51758400/0xef30c340bfb3ced3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6051 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571722.637360:0:77618:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:2.0:1788571722.637366:0:77618:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33a.replay-single-3, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), file ffff9ada606e6c00, open_flags 8042, mode 81a4 opened 0 00000080:00200000:2.0:1788571722.637376:0:77618:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33a.replay-single-3, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000002:00010000:2.0:1788571722.637403:0:77618:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33a.replay-single-3,[0x2000153f1:0x4:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:2.0:1788571722.637418:0:77618:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51758400/0xef30c340bfb3ced3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b82f6051 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571722.637488:0:77618:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51758400/0xef30c340bfb3ced3 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f6051 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571722.637496:0:77618:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1788571722.637499:0:77618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51758400/0xef30c340bfb3ced3 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f6051 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571722.637543:0:77618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8c00/0xef30c340bfb3ceda lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 77618 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.637550:0:77618:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8c00/0xef30c340bfb3ceda lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77618 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1788571722.637564:0:77618:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1788571722.637573:0:77618:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada60a35c00 x1875449613539200/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1788571722.637583:0:77618:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8c00/0xef30c340bfb3ceda lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77618 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571722.637612:0:77618:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35c00 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:77618:1875449613539200:192.168.202.113@tcp:101:createmany.0 00000100:00100000:2.0:1788571722.637816:0:77618:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada50f7a480 going to sleep for 11 seconds 00000100:00100000:0.0:1788571722.736325:0:77618:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada516f3480 x1875449613541760/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:0.0:1788571722.736333:0:77618:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3cf12 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77618 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571722.736353:0:77618:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f3480 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:77618:1875449613541760:192.168.202.113@tcp:101:createmany.0 00000100:00100000:0.0:1788571722.736569:0:77618:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada4522e180 going to sleep for 11 seconds 00000100:00100000:0.0:1788571722.742316:0:77618:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f3480 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:77618:1875449613541760:192.168.202.113@tcp:101:createmany.0 00010000:00010000:0.0:1788571722.742351:0:77618:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3cf12 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6131 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571722.742389:0:77618:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3cf12 lrc: 4/0,1 mode: --/PR res: [0x2000153f1:0x8:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6131 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571722.742399:0:77618:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3cf12 lrc: 4/0,1 mode: --/PR res: [0x2000153f1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6131 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571722.742413:0:77618:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3cf12 lrc: 4/0,1 mode: PR/PR res: [0x2000153f1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6131 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571722.742425:0:77618:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3cf12 lrc: 4/0,1 mode: PR/PR res: [0x2000153f1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6131 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571722.742443:0:77618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3cf12 lrc: 5/1,1 mode: PR/PR res: [0x2000153f1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6131 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571722.742466:0:77618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3cf12 lrc: 5/1,1 mode: PR/PR res: [0x2000153f1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6131 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571722.742480:0:77618:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3cf12 lrc: 4/1,0 mode: PR/PR res: [0x2000153f1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6131 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788571722.742494:0:77618:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9ada516f3480 x1875449613541760/t188978561044(188978561044) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/664 e 0 to 0 dl 1788571738 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:0.0:1788571722.742804:0:77618:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada516f3480 x1875449613541760/t188978561044(188978561044) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/664 e 0 to 0 dl 1788571738 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788571722.743367:0:77618:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3cf12 lrc: 3/1,0 mode: PR/PR res: [0x2000153f1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6131 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:0.0:1788571722.743388:0:77618:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3cf12 lrc: 3/1,0 mode: PR/PR res: [0x2000153f1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6131 expref: -99 pid: 77618 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571722.743403:0:77618:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8590021617/8 (0 0) 00000080:00200000:0.0:1788571722.743410:0:77618:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1788571722.743427:0:77618:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33a.replay-single-7, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000080:00200000:0.0:1788571722.743472:0:77618:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x2000153f1:0x8:0x0](ffff9ada5825aa08) 00000080:00200000:0.0:1788571722.743579:0:77618:0:(file.c:6843:ll_layout_conf()) [0x2000153f1:0x8:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:0.0:1788571722.743585:0:77618:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x2000153f1:0x8:0x0](ffff9ada5825aa08) 00000080:00010000:0.0:1788571722.743588:0:77618:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x2000153f1:0x8:0x0](ffff9ada5825aa08) for lock 0xef30c340bfb3cf12 00000080:00010000:0.0:1788571722.743601:0:77618:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x2000153f1:0x8:0x0](ffff9ada5825aa08) 00000080:00200008:0.0:1788571722.743619:0:77618:0:(file.c:1090:ll_file_open()) START file f33a.replay-single-7:[0x2000153f1:0x8:0x0](ffff9ada5825aa08), flags 100102 00000002:00100000:0.0:1788571722.743646:0:77618:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada516f3480 x1875449613541760/t188978561044(188978561044) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571738 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:0.0:1788571722.743709:0:77618:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cf12 from it ffff9ada517613c0 00010000:00010000:0.0:1788571722.743714:0:77618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3cf12 lrc: 3/1,0 mode: PR/PR res: [0x2000153f1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f6131 expref: -99 pid: 77618 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571722.743726:0:77618:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3cf12 lrc: 2/0,0 mode: PR/PR res: [0x2000153f1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f6131 expref: -99 pid: 77618 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571722.743753:0:77618:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x2000153f1:0x8:0x0](ffff9ada5825aa08), inode mode 81a4 mask 40 00000080:00200008:0.0:1788571722.743791:0:77618:0:(file.c:479:ll_file_release()) START file f33a.replay-single-7:[0x2000153f1:0x8:0x0](ffff9ada5825aa08), flags 100002 00000002:00100000:0.0:1788571722.743897:0:77618:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9ada516f3480 x1875449613541760/t188978561044(188978561044) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571738 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:0.0:1788571722.743910:0:77618:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:0.0:1788571722.743915:0:77618:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada516f3100 x1875449613542016/t0(0) o35->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:0.0:1788571722.743942:0:77618:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f3100 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:77618:1875449613542016:192.168.202.113@tcp:35:createmany.0 00000100:00100000:0.0:1788571722.744052:0:77618:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada4522e300 going to sleep for 11 seconds 00000100:00100000:0.0:1788571722.750789:0:77618:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f3100 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:77618:1875449613542016:192.168.202.113@tcp:35:createmany.0 00000002:00100000:0.0:1788571722.753560:0:77618:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9ada516f3480 x1875449613541760/t188978561044(188978561044) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571738 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1788571722.754610:0:77618:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571722.755229:0:77618:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571722.755317:0:77618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3cf19 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 77618 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.755329:0:77618:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3cf19 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77618 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.755339:0:77618:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3cf19 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77618 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571722.755371:0:77618:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35880 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:77618:1875449613542144:192.168.202.113@tcp:101:createmany.0 00000100:00100000:2.0:1788571722.755487:0:77618:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada50f7ae40 going to sleep for 11 seconds 00000100:00100000:2.0:1788571722.763210:0:77618:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a35880 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:77618:1875449613542144:192.168.202.113@tcp:101:createmany.0 00010000:00010000:2.0:1788571722.763247:0:77618:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3cf19 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6146 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.763263:0:77618:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3cf19 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6146 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.763278:0:77618:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3cf19 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6146 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.763290:0:77618:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3cf19 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6146 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.763305:0:77618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3cf19 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6146 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.763320:0:77618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3cf19 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6146 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.763328:0:77618:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3cf19 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6146 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1788571722.763340:0:77618:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada60a35880 x1875449613542144/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788571738 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1788571722.763356:0:77618:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3cf19 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6146 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.763373:0:77618:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:2.0:1788571722.763382:0:77618:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:2.0:1788571722.763419:0:77618:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:2.0:1788571722.763427:0:77618:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3cf19 00000080:00010000:2.0:1788571722.763433:0:77618:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571722.763437:0:77618:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cf19 from it ffffab0ec0d9bbf8 00010000:00010000:2.0:1788571722.763441:0:77618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3cf19 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6146 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.763461:0:77618:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3cf19 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6146 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571722.763492:0:77618:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:2.0:1788571722.763533:0:77618:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571722.763554:0:77618:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571722.763564:0:77618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3cf19 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6146 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571722.763576:0:77618:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3cf19 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6146 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571722.763583:0:77618:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571722.763589:0:77618:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571722.763593:0:77618:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3cf19 00000080:00010000:2.0:1788571722.763596:0:77618:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cf19 from it ffffab0ec0d9bb38 00010000:00010000:2.0:1788571722.763599:0:77618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3cf19 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6146 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571722.763607:0:77618:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3cf19 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6146 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571722.763615:0:77618:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:2.0:1788571722.763621:0:77618:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33a.replay-single-8, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), file ffff9ada600b2300, open_flags 8042, mode 81a4 opened 0 00000080:00200000:2.0:1788571722.763632:0:77618:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33a.replay-single-8, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000002:00010000:2.0:1788571722.763659:0:77618:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33a.replay-single-8,[0x2000153f1:0x9:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:2.0:1788571722.763673:0:77618:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3cf19 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b82f6146 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571722.763730:0:77618:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3cf19 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f6146 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571722.763738:0:77618:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1788571722.763741:0:77618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3cf19 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f6146 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571722.763789:0:77618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cf20 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 77618 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.763797:0:77618:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cf20 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77618 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1788571722.763812:0:77618:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1788571722.763820:0:77618:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada60a35880 x1875449613542400/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1788571722.763830:0:77618:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cf20 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77618 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571722.764722:0:77618:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35880 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:77618:1875449613542400:192.168.202.113@tcp:101:createmany.0 00000100:00100000:2.0:1788571722.764951:0:77618:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada50f7ae40 going to sleep for 11 seconds 00000100:00100000:2.0:1788571722.773180:0:77618:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a35880 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:77618:1875449613542400:192.168.202.113@tcp:101:createmany.0 00010000:00010000:2.0:1788571722.773218:0:77618:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cf20 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6162 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.773254:0:77618:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cf20 lrc: 4/0,1 mode: --/PR res: [0x2000153f1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6162 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.773263:0:77618:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cf20 lrc: 4/0,1 mode: --/PR res: [0x2000153f1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6162 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.773276:0:77618:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cf20 lrc: 4/0,1 mode: PR/PR res: [0x2000153f1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6162 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.773293:0:77618:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cf20 lrc: 4/0,1 mode: PR/PR res: [0x2000153f1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6162 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.773309:0:77618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cf20 lrc: 5/1,1 mode: PR/PR res: [0x2000153f1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6162 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.773318:0:77618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cf20 lrc: 5/1,1 mode: PR/PR res: [0x2000153f1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6162 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.773326:0:77618:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cf20 lrc: 4/1,0 mode: PR/PR res: [0x2000153f1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6162 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1788571722.773339:0:77618:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9ada60a35880 x1875449613542400/t188978561046(188978561046) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/664 e 0 to 0 dl 1788571738 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1788571722.773353:0:77618:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada60a35880 x1875449613542400/t188978561046(188978561046) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/664 e 0 to 0 dl 1788571738 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1788571722.773368:0:77618:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cf20 lrc: 3/1,0 mode: PR/PR res: [0x2000153f1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6162 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1788571722.773383:0:77618:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cf20 lrc: 3/1,0 mode: PR/PR res: [0x2000153f1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6162 expref: -99 pid: 77618 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.773396:0:77618:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8590021617/9 (0 0) 00000080:00200000:2.0:1788571722.773403:0:77618:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1788571722.773419:0:77618:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33a.replay-single-8, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000080:00200000:2.0:1788571722.773469:0:77618:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x2000153f1:0x9:0x0](ffff9ada58196c08) 00000080:00200000:2.0:1788571722.773566:0:77618:0:(file.c:6843:ll_layout_conf()) [0x2000153f1:0x9:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1788571722.773572:0:77618:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x2000153f1:0x9:0x0](ffff9ada58196c08) 00000080:00010000:2.0:1788571722.773576:0:77618:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x2000153f1:0x9:0x0](ffff9ada58196c08) for lock 0xef30c340bfb3cf20 00000080:00010000:2.0:1788571722.773588:0:77618:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x2000153f1:0x9:0x0](ffff9ada58196c08) 00000080:00200008:2.0:1788571722.773608:0:77618:0:(file.c:1090:ll_file_open()) START file f33a.replay-single-8:[0x2000153f1:0x9:0x0](ffff9ada58196c08), flags 100102 00000002:00100000:2.0:1788571722.773635:0:77618:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada60a35880 x1875449613542400/t188978561046(188978561046) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571738 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1788571722.773678:0:77618:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cf20 from it ffff9ada602e3c00 00010000:00010000:2.0:1788571722.773683:0:77618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cf20 lrc: 3/1,0 mode: PR/PR res: [0x2000153f1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f6162 expref: -99 pid: 77618 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.773694:0:77618:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cf20 lrc: 2/0,0 mode: PR/PR res: [0x2000153f1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f6162 expref: -99 pid: 77618 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571722.773722:0:77618:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x2000153f1:0x9:0x0](ffff9ada58196c08), inode mode 81a4 mask 40 00000080:00200008:2.0:1788571722.773762:0:77618:0:(file.c:479:ll_file_release()) START file f33a.replay-single-8:[0x2000153f1:0x9:0x0](ffff9ada58196c08), flags 100002 00000002:00100000:2.0:1788571722.773814:0:77618:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9ada60a35880 x1875449613542400/t188978561046(188978561046) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571738 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1788571722.773827:0:77618:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1788571722.773833:0:77618:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada60a35180 x1875449613542656/t0(0) o35->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1788571722.773885:0:77618:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35180 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:77618:1875449613542656:192.168.202.113@tcp:35:createmany.0 00000100:00100000:2.0:1788571722.773985:0:77618:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada50f7a900 going to sleep for 11 seconds 00000100:00100000:2.0:1788571722.778596:0:77618:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a35180 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:77618:1875449613542656:192.168.202.113@tcp:35:createmany.0 00000002:00100000:2.0:1788571722.778649:0:77618:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9ada60a35880 x1875449613542400/t188978561046(188978561046) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571738 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1788571722.778776:0:77618:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571722.778803:0:77618:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571722.778936:0:77618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3cf27 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 77618 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.778947:0:77618:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3cf27 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77618 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.778958:0:77618:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3cf27 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77618 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571722.778983:0:77618:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35500 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:77618:1875449613542784:192.168.202.113@tcp:101:createmany.0 00000100:00100000:2.0:1788571722.779083:0:77618:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada50f7a900 going to sleep for 11 seconds 00000100:00100000:2.0:1788571722.783032:0:77618:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a35500 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:77618:1875449613542784:192.168.202.113@tcp:101:createmany.0 00010000:00010000:2.0:1788571722.783078:0:77618:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3cf27 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6177 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.783094:0:77618:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3cf27 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6177 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.783109:0:77618:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3cf27 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6177 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.783121:0:77618:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3cf27 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6177 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.783142:0:77618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3cf27 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6177 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.783151:0:77618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3cf27 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6177 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.783159:0:77618:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3cf27 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6177 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1788571722.783173:0:77618:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada60a35500 x1875449613542784/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788571738 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1788571722.783191:0:77618:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3cf27 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6177 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.783207:0:77618:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:2.0:1788571722.783216:0:77618:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:2.0:1788571722.783257:0:77618:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:2.0:1788571722.783264:0:77618:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3cf27 00000080:00010000:2.0:1788571722.783271:0:77618:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571722.783276:0:77618:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cf27 from it ffffab0ec0d9bbf8 00010000:00010000:2.0:1788571722.783279:0:77618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3cf27 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6177 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.783289:0:77618:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3cf27 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6177 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571722.783326:0:77618:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:2.0:1788571722.783355:0:77618:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571722.783375:0:77618:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571722.783385:0:77618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3cf27 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6177 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571722.783397:0:77618:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3cf27 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6177 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571722.783405:0:77618:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571722.783409:0:77618:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571722.783413:0:77618:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3cf27 00000080:00010000:2.0:1788571722.783417:0:77618:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cf27 from it ffffab0ec0d9bb38 00010000:00010000:2.0:1788571722.783421:0:77618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3cf27 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6177 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571722.783428:0:77618:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3cf27 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6177 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571722.783437:0:77618:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:2.0:1788571722.783442:0:77618:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33a.replay-single-9, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), file ffff9ada467e8900, open_flags 8042, mode 81a4 opened 0 00000080:00200000:2.0:1788571722.783463:0:77618:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33a.replay-single-9, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000002:00010000:2.0:1788571722.783491:0:77618:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33a.replay-single-9,[0x2000153f1:0xa:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:2.0:1788571722.783506:0:77618:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3cf27 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b82f6177 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571722.783569:0:77618:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3cf27 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f6177 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571722.783577:0:77618:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1788571722.783580:0:77618:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3cf27 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f6177 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571722.783627:0:77618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf2e lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 77618 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.783636:0:77618:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf2e lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77618 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1788571722.783648:0:77618:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1788571722.783656:0:77618:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada60a35500 x1875449613543040/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1788571722.783666:0:77618:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf2e lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77618 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571722.783693:0:77618:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35500 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:77618:1875449613543040:192.168.202.113@tcp:101:createmany.0 00000100:00100000:2.0:1788571722.783928:0:77618:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada50f7a900 going to sleep for 11 seconds 00000100:00100000:2.0:1788571722.788928:0:77618:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a35500 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:77618:1875449613543040:192.168.202.113@tcp:101:createmany.0 00010000:00010000:2.0:1788571722.788962:0:77618:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf2e lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6193 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.788997:0:77618:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf2e lrc: 4/0,1 mode: --/PR res: [0x2000153f1:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6193 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.789018:0:77618:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf2e lrc: 4/0,1 mode: --/PR res: [0x2000153f1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6193 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.789031:0:77618:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf2e lrc: 4/0,1 mode: PR/PR res: [0x2000153f1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6193 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.789043:0:77618:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf2e lrc: 4/0,1 mode: PR/PR res: [0x2000153f1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6193 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.789058:0:77618:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf2e lrc: 5/1,1 mode: PR/PR res: [0x2000153f1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6193 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.789078:0:77618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf2e lrc: 5/1,1 mode: PR/PR res: [0x2000153f1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6193 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.789086:0:77618:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf2e lrc: 4/1,0 mode: PR/PR res: [0x2000153f1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6193 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1788571722.789100:0:77618:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9ada60a35500 x1875449613543040/t188978561048(188978561048) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/664 e 0 to 0 dl 1788571738 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1788571722.789113:0:77618:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada60a35500 x1875449613543040/t188978561048(188978561048) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/664 e 0 to 0 dl 1788571738 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1788571722.789128:0:77618:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf2e lrc: 3/1,0 mode: PR/PR res: [0x2000153f1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6193 expref: -99 pid: 77618 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1788571722.789144:0:77618:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf2e lrc: 3/1,0 mode: PR/PR res: [0x2000153f1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6193 expref: -99 pid: 77618 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.789158:0:77618:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8590021617/10 (0 0) 00000080:00200000:2.0:1788571722.789165:0:77618:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1788571722.789181:0:77618:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33a.replay-single-9, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000080:00200000:2.0:1788571722.789221:0:77618:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x2000153f1:0xa:0x0](ffff9ada581900c8) 00000080:00200000:2.0:1788571722.789314:0:77618:0:(file.c:6843:ll_layout_conf()) [0x2000153f1:0xa:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1788571722.789319:0:77618:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x2000153f1:0xa:0x0](ffff9ada581900c8) 00000080:00010000:2.0:1788571722.789323:0:77618:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x2000153f1:0xa:0x0](ffff9ada581900c8) for lock 0xef30c340bfb3cf2e 00000080:00010000:2.0:1788571722.789335:0:77618:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x2000153f1:0xa:0x0](ffff9ada581900c8) 00000080:00200008:2.0:1788571722.789354:0:77618:0:(file.c:1090:ll_file_open()) START file f33a.replay-single-9:[0x2000153f1:0xa:0x0](ffff9ada581900c8), flags 100102 00000002:00100000:2.0:1788571722.789377:0:77618:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada60a35500 x1875449613543040/t188978561048(188978561048) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571738 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1788571722.789418:0:77618:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cf2e from it ffff9ada602e3c00 00010000:00010000:2.0:1788571722.789423:0:77618:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf2e lrc: 3/1,0 mode: PR/PR res: [0x2000153f1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f6193 expref: -99 pid: 77618 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571722.789434:0:77618:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf2e lrc: 2/0,0 mode: PR/PR res: [0x2000153f1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f6193 expref: -99 pid: 77618 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571722.789470:0:77618:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x2000153f1:0xa:0x0](ffff9ada581900c8), inode mode 81a4 mask 40 00000080:00200008:2.0:1788571722.789507:0:77618:0:(file.c:479:ll_file_release()) START file f33a.replay-single-9:[0x2000153f1:0xa:0x0](ffff9ada581900c8), flags 100002 00000002:00100000:2.0:1788571722.789562:0:77618:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9ada60a35500 x1875449613543040/t188978561048(188978561048) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571738 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1788571722.789574:0:77618:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1788571722.789580:0:77618:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada60a37b80 x1875449613543296/t0(0) o35->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1788571722.789614:0:77618:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37b80 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:77618:1875449613543296:192.168.202.113@tcp:35:createmany.0 00000100:00100000:2.0:1788571722.789721:0:77618:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada50f7ad80 going to sleep for 11 seconds 00000100:00100000:2.0:1788571722.796544:0:77618:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37b80 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:77618:1875449613543296:192.168.202.113@tcp:35:createmany.0 00000002:00100000:2.0:1788571722.796590:0:77618:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9ada60a35500 x1875449613543040/t188978561048(188978561048) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571738 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00010000:00010000:0.0:1788571723.647729:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788571724.663714:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788571725.177160:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a49f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449613543424:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571725.177293:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a49c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613543680:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571725.177321:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a4bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613543936:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571725.177350:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1788571725.177355:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a49180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613543552:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571725.177374:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a48380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613543808:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571725.197143:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a49f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449613543424:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571725.204320:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a49180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613543552:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571725.204336:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a48380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613543808:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571725.204346:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a49c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613543680:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571725.204355:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613543936:192.168.202.113@tcp:400:kworker.0 00010000:00010000:2.0:1788571725.693162:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788571726.716025:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788571727.735010:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571728.765155:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788571729.785961:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788571730.297237:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449613544064:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571730.297390:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613544320:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571730.297426:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613544576:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571730.297463:0:2383:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1788571730.297469:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613544192:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571730.297517:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613544448:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571730.316654:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613544192:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571730.316756:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613544448:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571730.316766:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449613544064:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571730.316775:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613544320:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571730.316785:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613544576:192.168.202.113@tcp:400:kworker.0 00010000:00010000:2.0:1788571731.831765:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788571732.855367:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571734.907031:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788571735.415274:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a4bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449613544704:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571735.416111:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a49c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613544832:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571735.416228:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a49180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613545088:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571735.416271:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1788571735.416278:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a48380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613544960:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571735.416305:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a49f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613545216:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571735.432720:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a49500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449613545344:192.168.202.113@tcp:38:ptlrpcd_01_00.0 00010000:00010000:0.0:1788571735.927193:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571737.975155:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788571738.999709:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788571740.535211:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788571740.535219:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788571740.535224:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788571740.535238:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 3653 00000100:00080000:0.0:1788571740.535253:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 5s 00000100:00080000:0.0:1788571740.535262:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788571740.535276:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788571740.535298:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada516f0380 x1875449613545600/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788571740.535385:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613545728:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571740.535517:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613545984:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571740.535575:0:2383:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1788571740.535582:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449613545472:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571740.535618:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613545856:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571740.556295:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613545728:192.168.202.113@tcp:400:kworker.0 00010000:00010000:2.0:1788571741.047063:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788571742.080321:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571744.119043:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788571745.143186:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788571745.656471:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a49180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449613546240:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788571745.659972:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a49500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449613546112:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571745.660114:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a49f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613546496:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571745.660158:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1788571745.660170:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a49c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613546368:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571745.660207:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a48380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613546624:192.168.202.113@tcp:400:kworker.0 00000100:00080000:0.0:1788571745.661088:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788571745.661102:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-19) 00000100:00100000:0.0:1788571745.661109:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a49180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449613546240:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788571745.672242:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a49c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613546368:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571745.672263:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a48380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613546624:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571745.672276:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a49f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613546496:192.168.202.113@tcp:400:kworker.0 00010000:00010000:2.0:1788571747.191158:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788571748.222484:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571750.265433:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788571750.788335:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a4a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449613546880:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788571750.788482:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a48a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449613546752:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571750.790499:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a49880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613547136:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571750.791487:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1788571750.791505:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a4ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613547008:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571750.791662:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a48000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613547264:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571750.799251:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613547008:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571750.799275:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a48000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613547264:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571750.799288:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a49880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613547136:192.168.202.113@tcp:400:kworker.0 00000100:00080000:2.0:1788571750.799299:0:2385:0:(recover.c:205:ptlrpc_request_handle_notconn()) import MGC192.168.202.113@tcp of MGS@192.168.202.113@tcp abruptly disconnected: reconnecting 00000100:00080000:2.0:1788571750.799315:0:2385:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from FULL to DISCONN 00000100:02020000:2.0:1788571750.799321:0:2385:0:(import.c:186:ptlrpc_set_import_discon()) MGC192.168.202.113@tcp: Connection to MGS (at 192.168.202.113@tcp) was lost; in progress operations using this service will fail 00000100:00080000:2.0:1788571750.799325:0:2385:0:(import.c:192:ptlrpc_set_import_discon()) import MGS@192.168.202.113@tcp for MGC192.168.202.113@tcp not replayable, auto-deactivating 00000100:00080000:2.0:1788571750.799328:0:2385:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import MGS INVALID 00000100:00100000:2.0:1788571750.799336:0:2385:0:(client.c:3591:ptlrpc_abort_inflight()) @@@ inflight req@ffff9ada60a4bb80 x1875449613544704/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788571751 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788571750.799355:0:2385:0:(client.c:3591:ptlrpc_abort_inflight()) @@@ inflight req@ffff9ada516f1c00 x1875449613545472/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788571756 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788571750.799375:0:2385:0:(client.c:3591:ptlrpc_abort_inflight()) @@@ inflight req@ffff9ada60a49500 x1875449613546112/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788571761 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788571750.799627:0:2385:0:(client.c:3591:ptlrpc_abort_inflight()) @@@ inflight req@ffff9ada60a48a80 x1875449613546752/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788571766 ref 1 fl Rpc:RNQU/200/0 rc 0/-107 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:2.0:1788571750.799637:0:2385:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808001 00000100:00080000:2.0:1788571750.799641:0:2385:0:(pinger.c:193:ptlrpc_pinger_ir_down()) IR down 10000000:01000000:2.0:1788571750.799643:0:2385:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808002 00000100:00080000:2.0:1788571750.814489:0:2385:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788571750.814499:0:2385:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 3594 00000100:00080000:2.0:1788571750.814517:0:2385:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788571750.815059:0:2385:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60a37100 x1875449613547392/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788571750.815083:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a48a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449613546752:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571750.815097:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a49500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449613546112:192.168.202.113@tcp:400:kworker.0 00000100:00080000:2.0:1788571750.864758:0:77938:0:(import.c:1642:ptlrpc_invalidate_import_thread()) thread invalidate import lustre-MDT0000-mdc-ffff9ada58b0a000 to lustre-MDT0000_UUID@192.168.202.113@tcp 00000100:00080000:2.0:1788571750.864765:0:77938:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import lustre-MDT0000_UUID INVALID 00000100:00100000:2.0:1788571750.864769:0:77938:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 188978561028 gen 5 00000100:00100000:2.0:1788571750.864780:0:77938:0:(client.c:3164:ptlrpc_free_committed()) @@@ free request with old gen req@ffff9ada516f1880 x1875449613537280/t188978561030(188978561030) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571738 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000100:00100000:2.0:1788571750.864812:0:77938:0:(client.c:3164:ptlrpc_free_committed()) @@@ free request with old gen req@ffff9ada516f1180 x1875449613537536/t188978561031(188978561031) o35->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:23/10 lens 392/456 e 0 to 0 dl 1788571738 ref 1 fl Complete:RQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000100:00100000:2.0:1788571750.864827:0:77938:0:(client.c:3164:ptlrpc_free_committed()) @@@ free request with old gen req@ffff9ada60a4b100 x1875449613537920/t188978561032(188978561032) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571738 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000100:00100000:2.0:1788571750.864840:0:77938:0:(client.c:3164:ptlrpc_free_committed()) @@@ free request with old gen req@ffff9ada516f2d80 x1875449613538176/t188978561033(188978561033) o35->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:23/10 lens 392/456 e 0 to 0 dl 1788571738 ref 1 fl Complete:RQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000100:00100000:2.0:1788571750.864884:0:77938:0:(client.c:3164:ptlrpc_free_committed()) @@@ free request with old gen req@ffff9ada60a4b480 x1875449613538560/t188978561034(188978561034) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571738 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000100:00100000:2.0:1788571750.864898:0:77938:0:(client.c:3164:ptlrpc_free_committed()) @@@ free request with old gen req@ffff9ada516f3800 x1875449613538816/t188978561035(188978561035) o35->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:23/10 lens 392/456 e 0 to 0 dl 1788571738 ref 1 fl Complete:RQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000100:00100000:2.0:1788571750.864910:0:77938:0:(client.c:3164:ptlrpc_free_committed()) @@@ free request with old gen req@ffff9ada60a35c00 x1875449613539200/t188978561036(188978561036) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571738 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000100:00100000:2.0:1788571750.864924:0:77938:0:(client.c:3164:ptlrpc_free_committed()) @@@ free request with old gen req@ffff9ada516f0000 x1875449613539456/t188978561037(188978561037) o35->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:23/10 lens 392/456 e 0 to 0 dl 1788571738 ref 1 fl Complete:RQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000100:00100000:2.0:1788571750.864936:0:77938:0:(client.c:3164:ptlrpc_free_committed()) @@@ free request with old gen req@ffff9ada603f5c00 x1875449613539840/t188978561038(188978561038) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571738 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000100:00100000:2.0:1788571750.865341:0:77938:0:(client.c:3164:ptlrpc_free_committed()) @@@ free request with old gen req@ffff9ada603f7b80 x1875449613540096/t188978561039(188978561039) o35->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:23/10 lens 392/456 e 0 to 0 dl 1788571738 ref 1 fl Complete:RQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000100:00100000:2.0:1788571750.865356:0:77938:0:(client.c:3164:ptlrpc_free_committed()) @@@ free request with old gen req@ffff9ada516f1f80 x1875449613540480/t188978561040(188978561040) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571738 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000100:00100000:2.0:1788571750.865367:0:77938:0:(client.c:3164:ptlrpc_free_committed()) @@@ free request with old gen req@ffff9ada516f2680 x1875449613540736/t188978561041(188978561041) o35->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:23/10 lens 392/456 e 0 to 0 dl 1788571738 ref 1 fl Complete:RQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000100:00100000:2.0:1788571750.871041:0:77938:0:(client.c:3164:ptlrpc_free_committed()) @@@ free request with old gen req@ffff9ada516f0e00 x1875449613541120/t188978561042(188978561042) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571738 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000100:00100000:2.0:1788571750.871086:0:77938:0:(client.c:3164:ptlrpc_free_committed()) @@@ free request with old gen req@ffff9ada516f2a00 x1875449613541376/t188978561043(188978561043) o35->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:23/10 lens 392/456 e 0 to 0 dl 1788571738 ref 1 fl Complete:RQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000100:00100000:2.0:1788571750.871100:0:77938:0:(client.c:3164:ptlrpc_free_committed()) @@@ free request with old gen req@ffff9ada516f3480 x1875449613541760/t188978561044(188978561044) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571738 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000100:00100000:2.0:1788571750.871111:0:77938:0:(client.c:3164:ptlrpc_free_committed()) @@@ free request with old gen req@ffff9ada516f3100 x1875449613542016/t188978561045(188978561045) o35->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:23/10 lens 392/456 e 0 to 0 dl 1788571738 ref 1 fl Complete:RQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000100:00100000:2.0:1788571750.871122:0:77938:0:(client.c:3164:ptlrpc_free_committed()) @@@ free request with old gen req@ffff9ada60a35880 x1875449613542400/t188978561046(188978561046) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571738 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000100:00100000:2.0:1788571750.871135:0:77938:0:(client.c:3164:ptlrpc_free_committed()) @@@ free request with old gen req@ffff9ada60a35180 x1875449613542656/t188978561047(188978561047) o35->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:23/10 lens 392/456 e 0 to 0 dl 1788571738 ref 1 fl Complete:RQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000100:00100000:2.0:1788571750.871145:0:77938:0:(client.c:3164:ptlrpc_free_committed()) @@@ free request with old gen req@ffff9ada60a35500 x1875449613543040/t188978561048(188978561048) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571738 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000100:00100000:2.0:1788571750.871156:0:77938:0:(client.c:3164:ptlrpc_free_committed()) @@@ free request with old gen req@ffff9ada60a37b80 x1875449613543296/t188978561049(188978561049) o35->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:23/10 lens 392/456 e 0 to 0 dl 1788571738 ref 1 fl Complete:RQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000100:00100000:2.0:1788571750.871197:0:77938:0:(import.c:323:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 00010000:00010000:2.0:1788571750.871469:0:77938:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3ceb0 lrc: 3/0,0 mode: PR/PR res: [0x2000153f1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x12e400000000 nid: local remote: 0xdb7edb52b82f5fda expref: -99 pid: 77618 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571750.871502:0:77938:0:(vvp_object.c:117:vvp_conf_set()) [0x2000153f1:0x1:0x0]: losing layout lock 00010000:00010000:2.0:1788571750.871538:0:77938:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571750.871563:0:77938:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3ceb0 lrc: 0/0,0 mode: --/PR res: [0x2000153f1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x492f400000000 nid: local remote: 0xdb7edb52b82f5fda expref: -99 pid: 77618 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571750.871604:0:77938:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759600/0xef30c340bfb3cebe lrc: 3/0,0 mode: PR/PR res: [0x2000153f1:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x12e400000000 nid: local remote: 0xdb7edb52b82f600b expref: -99 pid: 77618 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571750.871616:0:77938:0:(vvp_object.c:117:vvp_conf_set()) [0x2000153f1:0x2:0x0]: losing layout lock 00010000:00010000:2.0:1788571750.871630:0:77938:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571750.871634:0:77938:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759600/0xef30c340bfb3cebe lrc: 0/0,0 mode: --/PR res: [0x2000153f1:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x492f400000000 nid: local remote: 0xdb7edb52b82f600b expref: -99 pid: 77618 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571750.871668:0:77938:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51758e00/0xef30c340bfb3cecc lrc: 3/0,0 mode: PR/PR res: [0x2000153f1:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x12e400000000 nid: local remote: 0xdb7edb52b82f603c expref: -99 pid: 77618 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571750.871679:0:77938:0:(vvp_object.c:117:vvp_conf_set()) [0x2000153f1:0x3:0x0]: losing layout lock 00010000:00010000:2.0:1788571750.871692:0:77938:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571750.871697:0:77938:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51758e00/0xef30c340bfb3cecc lrc: 0/0,0 mode: --/PR res: [0x2000153f1:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x492f400000000 nid: local remote: 0xdb7edb52b82f603c expref: -99 pid: 77618 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571750.871727:0:77938:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8c00/0xef30c340bfb3ceda lrc: 3/0,0 mode: PR/PR res: [0x2000153f1:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x12e400000000 nid: local remote: 0xdb7edb52b82f606d expref: -99 pid: 77618 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571750.871737:0:77938:0:(vvp_object.c:117:vvp_conf_set()) [0x2000153f1:0x4:0x0]: losing layout lock 00010000:00010000:2.0:1788571750.871747:0:77938:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571750.871751:0:77938:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8c00/0xef30c340bfb3ceda lrc: 0/0,0 mode: --/PR res: [0x2000153f1:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x492f400000000 nid: local remote: 0xdb7edb52b82f606d expref: -99 pid: 77618 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571750.871784:0:77938:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f200/0xef30c340bfb3cee8 lrc: 3/0,0 mode: PR/PR res: [0x2000153f1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x12e400000000 nid: local remote: 0xdb7edb52b82f609e expref: -99 pid: 77618 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571750.871793:0:77938:0:(vvp_object.c:117:vvp_conf_set()) [0x2000153f1:0x5:0x0]: losing layout lock 00010000:00010000:2.0:1788571750.871804:0:77938:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571750.871808:0:77938:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f200/0xef30c340bfb3cee8 lrc: 0/0,0 mode: --/PR res: [0x2000153f1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x492f400000000 nid: local remote: 0xdb7edb52b82f609e expref: -99 pid: 77618 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571750.871840:0:77938:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616800/0xef30c340bfb3cef6 lrc: 3/0,0 mode: PR/PR res: [0x2000153f1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x12e400000000 nid: local remote: 0xdb7edb52b82f60cf expref: -99 pid: 77618 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571750.871881:0:77938:0:(vvp_object.c:117:vvp_conf_set()) [0x2000153f1:0x6:0x0]: losing layout lock 00010000:00010000:2.0:1788571750.871893:0:77938:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571750.871897:0:77938:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616800/0xef30c340bfb3cef6 lrc: 0/0,0 mode: --/PR res: [0x2000153f1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x492f400000000 nid: local remote: 0xdb7edb52b82f60cf expref: -99 pid: 77618 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571750.871930:0:77938:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617000/0xef30c340bfb3cf04 lrc: 3/0,0 mode: PR/PR res: [0x2000153f1:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x12e400000000 nid: local remote: 0xdb7edb52b82f6100 expref: -99 pid: 77618 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571750.871951:0:77938:0:(vvp_object.c:117:vvp_conf_set()) [0x2000153f1:0x7:0x0]: losing layout lock 00010000:00010000:2.0:1788571750.871962:0:77938:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571750.871967:0:77938:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617000/0xef30c340bfb3cf04 lrc: 0/0,0 mode: --/PR res: [0x2000153f1:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x492f400000000 nid: local remote: 0xdb7edb52b82f6100 expref: -99 pid: 77618 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571750.872000:0:77938:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3cf12 lrc: 3/0,0 mode: PR/PR res: [0x2000153f1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x12e400000000 nid: local remote: 0xdb7edb52b82f6131 expref: -99 pid: 77618 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571750.872010:0:77938:0:(vvp_object.c:117:vvp_conf_set()) [0x2000153f1:0x8:0x0]: losing layout lock 00010000:00010000:2.0:1788571750.872020:0:77938:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571750.872024:0:77938:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3cf12 lrc: 0/0,0 mode: --/PR res: [0x2000153f1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x492f400000000 nid: local remote: 0xdb7edb52b82f6131 expref: -99 pid: 77618 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571750.872052:0:77938:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cf20 lrc: 3/0,0 mode: PR/PR res: [0x2000153f1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x12e400000000 nid: local remote: 0xdb7edb52b82f6162 expref: -99 pid: 77618 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571750.872062:0:77938:0:(vvp_object.c:117:vvp_conf_set()) [0x2000153f1:0x9:0x0]: losing layout lock 00010000:00010000:2.0:1788571750.872071:0:77938:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571750.872076:0:77938:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cf20 lrc: 0/0,0 mode: --/PR res: [0x2000153f1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x492f400000000 nid: local remote: 0xdb7edb52b82f6162 expref: -99 pid: 77618 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571750.872110:0:77938:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf2e lrc: 3/0,0 mode: PR/PR res: [0x2000153f1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x12e400000000 nid: local remote: 0xdb7edb52b82f6193 expref: -99 pid: 77618 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571750.872121:0:77938:0:(vvp_object.c:117:vvp_conf_set()) [0x2000153f1:0xa:0x0]: losing layout lock 00010000:00010000:2.0:1788571750.872132:0:77938:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571750.872137:0:77938:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf2e lrc: 0/0,0 mode: --/PR res: [0x2000153f1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x492f400000000 nid: local remote: 0xdb7edb52b82f6193 expref: -99 pid: 77618 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000100:00080000:2.0:1788571750.883489:0:77938:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from EVICTED to RECOVER 00000100:00080000:2.0:1788571750.883505:0:77938:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from RECOVER to FULL 00000100:00080000:2.0:1788571750.887879:0:2382:0:(import.c:1176:ptlrpc_connect_interpret()) MGC192.168.202.113@tcp: connect to target with instance 0 10000000:01000000:2.0:1788571750.887906:0:2382:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808005 00000100:02000400:2.0:1788571750.887918:0:2382:0:(import.c:1328:ptlrpc_connect_interpret()) Evicted from MGS (at 192.168.202.113@tcp) after server handle changed from 0xdb7edb52b82f5aee to 0xdb7edb52b82f6560 00000100:00080000:2.0:1788571750.887924:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to EVICTED 00000100:00080000:2.0:1788571750.887932:0:2382:0:(import.c:1718:ptlrpc_import_recovery_state_machine()) evicted from MGS@192.168.202.113@tcp; invalidating 00000100:02000000:0.0:1788571750.894274:0:77938:0:(import.c:1787:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff9ada58b0a000: Connection restored to 192.168.202.113@tcp (at 192.168.202.113@tcp) 00010000:00010000:0.0:1788571751.293464:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:01000000:0.0:1788571752.746621:0:15207:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 52-51 (rc=0) 10000000:01000000:0.0:1788571752.746650:0:15207:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.202.113@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:0.0:1788571752.746667:0:15207:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.202.113@tcp lock: ffff9ada5163fa00/0xef30c340bfb3cf3c lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b82f656e expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571752.746681:0:15207:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.202.113@tcp lock: ffff9ada5163fa00/0xef30c340bfb3cf3c lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b82f656e expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1788571752.746694:0:15207:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:0.0:1788571752.746698:0:15207:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9ada58b0a000 from 7 10000000:01000000:0.0:1788571752.746701:0:15207:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:0.0:1788571752.746770:0:15207:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.202.113@tcp lock: ffff9ada51617e00/0xef30c340bfb3cf43 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571752.746779:0:15207:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.202.113@tcp lock: ffff9ada51617e00/0xef30c340bfb3cf43 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571752.746793:0:15207:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.202.113@tcp lock: ffff9ada51617e00/0xef30c340bfb3cf43 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571752.746830:0:15207:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0a80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449613548160:192.168.202.113@tcp:101:ll_cfg_requeue.0 00000100:00100000:0.0:1788571752.746991:0:15207:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada4522e300 going to sleep for 11 seconds 00000100:00100000:0.0:1788571752.755545:0:15207:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0a80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449613548160:192.168.202.113@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1788571752.755596:0:15207:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.202.113@tcp lock: ffff9ada51617e00/0xef30c340bfb3cf43 lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b82f6575 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571752.755612:0:15207:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.202.113@tcp lock: ffff9ada51617e00/0xef30c340bfb3cf43 lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0xdb7edb52b82f6575 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571752.755715:0:15207:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0a80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449613548288:192.168.202.113@tcp:501:ll_cfg_requeue.0 00000100:00100000:0.0:1788571752.755817:0:15207:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada4522e300 going to sleep for 11 seconds 00000100:00100000:0.0:1788571752.785732:0:15207:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0a80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449613548288:192.168.202.113@tcp:501:ll_cfg_requeue.0 00000100:00100000:0.0:1788571752.785827:0:15207:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0a80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449613548416:192.168.202.113@tcp:503:ll_cfg_requeue.0 00000100:00100000:0.0:1788571752.785987:0:15207:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada4522e300 going to sleep for 11 seconds 00000100:00100000:0.0:1788571752.796919:0:15207:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0a80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449613548416:192.168.202.113@tcp:503:ll_cfg_requeue.0 00000040:00080000:0.0:1788571752.800359:0:77948:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x2:0x0] index 64768 count 7 00010000:00010000:2.0:1788571753.336362:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788571754.367741:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788571755.909679:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449613548800:192.168.202.113@tcp:400:kworker.0 00010000:00080000:2.0:1788571755.909715:0:2386:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff9ada60a4bb80 x1875449613548928/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788571771 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788571755.909730:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613548928:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571755.913972:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613549056:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571755.914010:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613549312:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571755.917625:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a48700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613549184:192.168.202.113@tcp:400:kworker.0 00010000:00010000:0.0:1788571757.431081:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:2.0:1788571760.057304:0:77999:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1788571760.057318:0:77999:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571760.057343:0:77999:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571760.059984:0:77999:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf51 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 77999 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571760.060003:0:77999:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf51 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77999 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571760.060015:0:77999:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf51 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77999 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571760.060046:0:77999:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37b80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:77999:1875449613549440:192.168.202.113@tcp:101:lfs.0 00000100:00100000:2.0:1788571760.060146:0:77999:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada53736d80 going to sleep for 11 seconds 00000100:00100000:2.0:1788571760.092301:0:77999:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37b80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:77999:1875449613549440:192.168.202.113@tcp:101:lfs.0 00010000:00010000:2.0:1788571760.092330:0:77999:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf51 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f65ad expref: -99 pid: 77999 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571760.092344:0:77999:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf51 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f65ad expref: -99 pid: 77999 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571760.092355:0:77999:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf51 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f65ad expref: -99 pid: 77999 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571760.092367:0:77999:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf51 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f65ad expref: -99 pid: 77999 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571760.092380:0:77999:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf51 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f65ad expref: -99 pid: 77999 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571760.092400:0:77999:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf51 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f65ad expref: -99 pid: 77999 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571760.092407:0:77999:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf51 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f65ad expref: -99 pid: 77999 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1788571760.092419:0:77999:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada60a37b80 x1875449613549440/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788571776 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1788571760.092433:0:77999:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf51 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f65ad expref: -99 pid: 77999 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571760.092446:0:77999:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:2.0:1788571760.092453:0:77999:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:2.0:1788571760.092481:0:77999:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:2.0:1788571760.092487:0:77999:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3cf51 00000080:00010000:2.0:1788571760.092493:0:77999:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571760.092496:0:77999:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cf51 from it ffffab0ec0d4bcb8 00010000:00010000:2.0:1788571760.092500:0:77999:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf51 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f65ad expref: -99 pid: 77999 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571760.092508:0:77999:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf51 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f65ad expref: -99 pid: 77999 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:2.0:1788571760.093260:0:77999:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 0, flags 0, foreign 0 00000080:00200000:2.0:1788571760.093268:0:77999:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571760.093286:0:77999:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571760.093298:0:77999:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf51 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f65ad expref: -99 pid: 77999 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571760.093310:0:77999:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf51 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f65ad expref: -99 pid: 77999 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571760.093317:0:77999:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571760.093322:0:77999:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571760.093325:0:77999:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3cf51 00000080:00010000:2.0:1788571760.093330:0:77999:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cf51 from it ffffab0ec0d4bcb8 00010000:00010000:2.0:1788571760.093334:0:77999:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf51 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f65ad expref: -99 pid: 77999 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571760.093342:0:77999:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf51 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f65ad expref: -99 pid: 77999 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571760.093452:0:77999:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571760.093461:0:77999:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571760.093469:0:77999:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf51 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f65ad expref: -99 pid: 77999 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571760.093478:0:77999:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf51 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f65ad expref: -99 pid: 77999 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571760.093484:0:77999:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571760.093488:0:77999:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571760.093491:0:77999:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3cf51 00000080:00010000:2.0:1788571760.093495:0:77999:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cf51 from it ffffab0ec0d4bbb0 00010000:00010000:2.0:1788571760.093499:0:77999:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf51 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f65ad expref: -99 pid: 77999 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571760.093507:0:77999:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf51 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f65ad expref: -99 pid: 77999 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1788571760.093516:0:77999:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:2.0:1788571760.093523:0:77999:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200080:2.0:1788571760.093638:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.093652:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00800000:00000004:2.0:1788571760.093658:0:77999:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: age 3632, max_age 3676 00000100:00100000:2.0:1788571760.093694:0:77999:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37b80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:77999:1875449613549568:192.168.202.113@tcp:41:lfs.0 00000100:00100000:2.0:1788571760.093787:0:77999:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada53736d80 going to sleep for 11 seconds 00000100:00100000:2.0:1788571760.099447:0:77999:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37b80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:77999:1875449613549568:192.168.202.113@tcp:41:lfs.0 00800000:00000004:2.0:1788571760.099478:0:77999:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: update ffff9ada4481d198 cache blocks 317380/353529 objects 1023693/1024000 00000080:00200080:2.0:1788571760.099588:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.099610:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00800000:00000004:2.0:1788571760.099619:0:77999:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: age 3632, max_age 3676 00000100:00100000:2.0:1788571760.099658:0:77999:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37b80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:77999:1875449613549696:192.168.202.113@tcp:41:lfs.0 00000100:00100000:2.0:1788571760.100013:0:77999:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada53736d80 going to sleep for 11 seconds 00000100:00100000:2.0:1788571760.106348:0:77999:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37b80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:77999:1875449613549696:192.168.202.113@tcp:41:lfs.0 00800000:00000004:2.0:1788571760.106381:0:77999:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: update ffff9ada44819e18 cache blocks 317424/353529 objects 1023740/1024000 00000080:00200080:2.0:1788571760.106465:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106481:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106490:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106495:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106499:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106503:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106506:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106510:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106514:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106517:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106521:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106525:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106528:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106531:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106536:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106539:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106543:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106547:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106550:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106554:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106558:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106562:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106567:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106570:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106586:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106591:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106597:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106602:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106608:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106612:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106617:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106620:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106624:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106628:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106631:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106635:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106640:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106643:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106648:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106652:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106657:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106660:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106665:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106669:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106673:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106677:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106681:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106684:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106688:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106690:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106694:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106697:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106702:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106705:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106709:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106712:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106716:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106719:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106724:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106728:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106734:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106738:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106743:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106749:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106752:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106757:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106761:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106766:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106769:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106772:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106776:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106780:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106784:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106788:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106793:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106798:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106803:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106807:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106813:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106818:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106825:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106830:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106834:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106838:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106844:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.106848:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.106886:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.107125:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.107138:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.107145:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.107150:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.107154:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.107159:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.107163:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.107168:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.107173:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.107178:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.107181:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.107185:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.107245:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.107255:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.107262:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.107267:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.107271:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.107275:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.107278:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.107281:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.107285:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.107289:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.107295:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.107301:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.107304:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.107308:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.107311:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.107315:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.107318:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.107322:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.107325:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.107329:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.107332:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.107337:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.107341:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.107345:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.107348:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.107351:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00800000:00000080:2.0:1788571760.107355:0:77999:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00000080:00200080:2.0:1788571760.107359:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:2.0:1788571760.107364:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653dc00 uarg=0000000000000000 00020000:00000004:2.0:1788571760.107375:0:77999:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: age 3632, max_age 3676 00000100:00100000:2.0:1788571760.107425:0:77999:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37b80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:77999:1875449613549824:192.168.202.113@tcp:13:lfs.0 00000100:00100000:2.0:1788571760.107527:0:77999:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada53736d80 going to sleep for 11 seconds 00000100:00100000:0.0:1788571760.124430:0:77999:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37b80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:77999:1875449613549824:192.168.202.113@tcp:13:lfs.0 00020000:00000004:0.0:1788571760.124473:0:77999:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: update ffff9ada4354d198 cache blocks 901357/958279 objects 261729/262144 00000080:00200080:0.0:1788571760.124693:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.124720:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00020000:00000004:0.0:1788571760.124731:0:77999:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: age 3632, max_age 3676 00000100:00100000:0.0:1788571760.124789:0:77999:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0a80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:77999:1875449613549952:192.168.202.113@tcp:13:lfs.0 00000100:00100000:0.0:1788571760.124922:0:77999:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60f806c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788571760.137755:0:77999:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0a80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:77999:1875449613549952:192.168.202.113@tcp:13:lfs.0 00020000:00000004:0.0:1788571760.137800:0:77999:0:(obd_class.h:1116:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: update ffff9ada43549e18 cache blocks 901363/958279 objects 261734/262144 00000080:00200080:0.0:1788571760.137922:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.137942:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.137948:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.137951:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.137954:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.137956:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.137959:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.137962:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.137964:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.137967:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.137969:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.137971:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.137974:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.137977:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140611:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140662:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140690:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140696:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140700:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140704:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140708:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140711:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140715:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140719:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140723:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140727:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140731:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140734:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140737:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140739:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140741:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140743:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140747:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140749:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140752:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140754:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140756:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140758:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140760:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140761:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140763:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140765:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140767:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140768:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140770:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140772:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140774:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140776:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140778:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140779:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140781:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140783:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140785:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140786:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140788:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140790:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140792:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140795:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140797:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140798:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140800:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140802:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140804:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140805:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140807:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140809:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140811:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140812:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140814:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140816:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140818:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140820:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140822:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140823:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140825:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140827:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140829:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140830:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140832:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140834:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140836:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140837:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140841:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140842:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140844:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140846:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140848:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140849:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140851:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140868:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140870:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140872:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140874:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140876:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140878:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140880:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140882:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140883:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140885:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140887:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140889:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140891:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140893:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140894:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140896:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140898:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140900:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140902:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140904:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140906:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140908:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140910:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140912:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140913:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140915:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140917:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140919:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140920:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140922:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140924:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140926:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140928:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140930:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140931:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571760.140933:0:77999:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd86985710 00020000:00000080:0.0:1788571760.140935:0:77999:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200008:0.0:1788571760.140951:0:77999:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000100:00100000:0.0:1788571761.066000:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613550208:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571761.066024:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613550464:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571761.066051:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613550336:192.168.202.113@tcp:400:kworker.0 00010000:00010000:2.0:1788571762.553780:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200080:0.0:1788571762.582113:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.582156:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00800000:00000004:0.0:1788571762.582167:0:78052:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: age 3677, max_age 3679 00000100:00100000:0.0:1788571762.582243:0:78052:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0a80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:78052:1875449613550848:192.168.202.113@tcp:41:lfs.0 00000100:00100000:0.0:1788571762.582369:0:78052:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada51134a80 going to sleep for 11 seconds 00000100:00100000:0.0:1788571762.589087:0:78052:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0a80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:78052:1875449613550848:192.168.202.113@tcp:41:lfs.0 00800000:00000004:0.0:1788571762.589126:0:78052:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: update ffff9ada44819e18 cache blocks 317424/353529 objects 1023740/1024000 00000080:00200080:0.0:1788571762.589267:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589290:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589298:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589302:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589307:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589311:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589314:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589318:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589322:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589326:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589330:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589333:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589336:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589338:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589340:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589342:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589344:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589346:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589350:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589353:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589357:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589361:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589364:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589367:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589371:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589374:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589378:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589381:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589385:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589388:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589392:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589395:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589399:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589402:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589406:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589411:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589420:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589423:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589427:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589430:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589433:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589437:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589441:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589446:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589450:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589454:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589458:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589461:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589465:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589469:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589473:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589476:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589480:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589483:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589487:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589490:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589493:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589496:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589499:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589503:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589507:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589511:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589515:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589519:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589522:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589526:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589530:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589533:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589539:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589543:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589547:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589550:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589554:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589557:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589562:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589565:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589569:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589572:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589577:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589580:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589596:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589601:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589604:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589608:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589612:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589615:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589619:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589622:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589626:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589629:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589633:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589636:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589640:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589643:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589647:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589651:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589655:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589659:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589663:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589667:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589670:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589673:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589676:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589680:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589684:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589688:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589692:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589696:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589699:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589703:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589708:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589711:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589715:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589718:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589722:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589725:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589729:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589732:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589737:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589740:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589744:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589748:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589752:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589755:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589760:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00800000:00000080:0.0:1788571762.589763:0:78052:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00000080:00200080:0.0:1788571762.589767:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:0.0:1788571762.589774:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0800 uarg=0000000000000000 00020000:00000004:0.0:1788571762.589785:0:78052:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: age 3677, max_age 3679 00000100:00100000:0.0:1788571762.589844:0:78052:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0a80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:78052:1875449613550976:192.168.202.113@tcp:13:lfs.0 00000100:00100000:0.0:1788571762.589987:0:78052:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada51134a80 going to sleep for 11 seconds 00000100:00100000:2.0:1788571762.609467:0:78052:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4b800 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:78052:1875449613551104:192.168.202.113@tcp:13:lfs.0 00020000:00000004:2.0:1788571762.609513:0:78052:0:(obd_class.h:1116:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: update ffff9ada43549e18 cache blocks 901363/958279 objects 261734/262144 00000080:00200080:2.0:1788571762.609609:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609630:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609637:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609640:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609643:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609645:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609648:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609649:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609651:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609653:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609655:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609657:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609659:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609660:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609662:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609664:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609666:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609668:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609670:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609672:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609675:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609677:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609679:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609681:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609683:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609685:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609687:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609688:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609690:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609692:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609694:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609695:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609697:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609699:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609701:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609703:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609705:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609706:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609708:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609710:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609712:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609713:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609715:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609717:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609720:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609722:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609724:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609726:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609728:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609730:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609732:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609734:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609736:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609737:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609739:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609741:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609743:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609745:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609746:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609748:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609750:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609752:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609754:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609755:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609757:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609759:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609761:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609763:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609765:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609767:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609770:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609772:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609774:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609776:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609778:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609780:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609782:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609783:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609785:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609787:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609789:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609791:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609792:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609794:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609796:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609798:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609800:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609801:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609803:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609805:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609807:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609808:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609810:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609812:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609815:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609817:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609820:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609821:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.609823:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.609825:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.610542:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.610560:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.610572:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.610576:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.610581:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.610595:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.610601:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.610603:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.610606:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.610608:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.610610:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.610612:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.610614:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.610616:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.610618:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.610620:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.610622:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.610624:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.610626:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.610629:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.610631:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.610634:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.610636:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.610638:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200080:2.0:1788571762.610640:0:78052:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd6c20fd10 00020000:00000080:2.0:1788571762.610641:0:78052:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4653f400 uarg=0000000000000000 00000080:00200008:2.0:1788571762.610657:0:78052:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00010000:00010000:2.0:1788571765.623822:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788571766.135799:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613551360:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571766.135959:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613551616:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571766.135996:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:2.0:1788571766.136002:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449613551232:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571766.136033:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613551488:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571766.136058:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a34700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613551744:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571771.259123:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449613551872:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571771.259223:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613552128:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571771.259262:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613552384:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571771.259298:0:2383:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1788571771.259304:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613552000:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571771.259326:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613552256:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571771.279149:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613552000:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571771.279181:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613552256:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571771.279198:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613552128:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571771.279215:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613552384:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571771.279289:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449613551872:192.168.202.113@tcp:400:kworker.0 00000080:00200000:0.0:1788571771.845846:0:78192:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571771.845900:0:78192:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571771.845921:0:78192:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf51 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f65ad expref: -99 pid: 77999 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:0.0:1788571771.847026:0:78192:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf51 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f65ad expref: -99 pid: 77999 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000080:00200000:0.0:1788571771.847260:0:78192:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788571771.847272:0:78192:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571771.847415:0:78192:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3cf51 00000080:00010000:0.0:1788571771.847426:0:78192:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cf51 from it ffffab0ec1cdfbf8 00010000:00010000:0.0:1788571771.847431:0:78192:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf51 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f65ad expref: -99 pid: 77999 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:0.0:1788571771.847441:0:78192:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf51 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f65ad expref: -99 pid: 77999 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00000080:00200000:0.0:1788571771.847457:0:78192:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:0.0:1788571771.847488:0:78192:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571771.847501:0:78192:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571771.847513:0:78192:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf51 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f65ad expref: -99 pid: 77999 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00010000:00010000:0.0:1788571771.848452:0:78192:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf51 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f65ad expref: -99 pid: 77999 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00000080:00200000:0.0:1788571771.848464:0:78192:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788571771.848471:0:78192:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571771.848475:0:78192:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3cf51 00000080:00010000:0.0:1788571771.848489:0:78192:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cf51 from it ffffab0ec1cdfb38 00010000:00010000:0.0:1788571771.848492:0:78192:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf51 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f65ad expref: -99 pid: 77999 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00010000:00010000:0.0:1788571771.848501:0:78192:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf51 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f65ad expref: -99 pid: 77999 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00000080:00200000:0.0:1788571771.848512:0:78192:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:0.0:1788571771.848518:0:78192:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33a.replay-single--0, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), file ffff9ada6064e780, open_flags 8042, mode 81a4 opened 0 00000080:00200000:0.0:1788571771.848528:0:78192:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33a.replay-single--0, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000100:00100000:0.0:1788571771.848589:0:78192:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0700 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78192:1875449613552512:192.168.202.113@tcp:700:createmany.0 00000100:00100000:0.0:1788571771.848744:0:78192:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada465f4000 going to sleep for 11 seconds 00000100:00100000:0.0:1788571771.862555:0:78192:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0700 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78192:1875449613552512:192.168.202.113@tcp:700:createmany.0 00000002:00010000:0.0:1788571771.862599:0:78192:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33a.replay-single--0,[0x200015bc1:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:0.0:1788571771.862639:0:78192:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf51 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b82f65ad expref: -99 pid: 77999 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:0.0:1788571771.862709:0:78192:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf51 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f65ad expref: -99 pid: 77999 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:0.0:1788571771.862718:0:78192:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1788571771.862721:0:78192:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3cf51 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f65ad expref: -99 pid: 77999 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:0.0:1788571771.862767:0:78192:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617a00/0xef30c340bfb3cf58 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78192 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571771.862775:0:78192:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617a00/0xef30c340bfb3cf58 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78192 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:0.0:1788571771.862787:0:78192:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:0.0:1788571771.862796:0:78192:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada516f0700 x1875449613552768/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:0.0:1788571771.862805:0:78192:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617a00/0xef30c340bfb3cf58 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78192 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571771.862827:0:78192:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0700 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78192:1875449613552768:192.168.202.113@tcp:101:createmany.0 00000100:00100000:2.0:1788571771.863572:0:78192:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada465f4000 going to sleep for 11 seconds 00000100:00100000:0.0:1788571771.875663:0:78192:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0700 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78192:1875449613552768:192.168.202.113@tcp:101:createmany.0 00010000:00010000:0.0:1788571771.875692:0:78192:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617a00/0xef30c340bfb3cf58 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f65c9 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571771.875729:0:78192:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617a00/0xef30c340bfb3cf58 lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f65c9 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571771.875737:0:78192:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617a00/0xef30c340bfb3cf58 lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f65c9 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571771.875752:0:78192:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617a00/0xef30c340bfb3cf58 lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f65c9 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571771.875761:0:78192:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617a00/0xef30c340bfb3cf58 lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f65c9 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571771.875775:0:78192:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617a00/0xef30c340bfb3cf58 lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f65c9 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571771.875783:0:78192:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617a00/0xef30c340bfb3cf58 lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f65c9 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571771.875791:0:78192:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617a00/0xef30c340bfb3cf58 lrc: 4/1,0 mode: PR/PR res: [0x200015bc1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f65c9 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788571771.875801:0:78192:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9ada516f0700 x1875449613552768/t193273528321(193273528321) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/664 e 0 to 0 dl 1788571787 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:0.0:1788571771.875813:0:78192:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada516f0700 x1875449613552768/t193273528321(193273528321) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/664 e 0 to 0 dl 1788571787 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788571771.875826:0:78192:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617a00/0xef30c340bfb3cf58 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f65c9 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:0.0:1788571771.875838:0:78192:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617a00/0xef30c340bfb3cf58 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f65c9 expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571771.875848:0:78192:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8590023617/1 (0 0) 00000080:00200000:0.0:1788571771.875993:0:78192:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1788571771.876011:0:78192:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33a.replay-single--0, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000100:00100000:2.0:1788571771.941010:0:78192:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada537366c0 going to sleep for 11 seconds 00000002:00100000:2.0:1788571771.986411:0:78192:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9ada603f5f80 x1875449613554048/t193273528325(193273528325) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571787 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1788571771.986537:0:78192:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571771.986835:0:78192:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571771.987064:0:78192:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9800/0xef30c340bfb3cf7b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78192 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571771.987081:0:78192:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9800/0xef30c340bfb3cf7b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78192 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571771.987095:0:78192:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9800/0xef30c340bfb3cf7b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78192 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571771.987124:0:78192:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a34380 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78192:1875449613554432:192.168.202.113@tcp:101:createmany.0 00000100:00100000:2.0:1788571771.987246:0:78192:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada4137da80 going to sleep for 11 seconds 00000100:00100000:0.0:1788571771.995293:0:78192:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a34380 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78192:1875449613554432:192.168.202.113@tcp:101:createmany.0 00010000:00010000:0.0:1788571771.995334:0:78192:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9800/0xef30c340bfb3cf7b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6640 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571771.995350:0:78192:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9800/0xef30c340bfb3cf7b lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6640 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571771.995366:0:78192:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9800/0xef30c340bfb3cf7b lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6640 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571771.995380:0:78192:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9800/0xef30c340bfb3cf7b lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6640 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571771.995396:0:78192:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9800/0xef30c340bfb3cf7b lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6640 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571771.995404:0:78192:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9800/0xef30c340bfb3cf7b lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6640 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571771.995414:0:78192:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9800/0xef30c340bfb3cf7b lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6640 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788571771.995429:0:78192:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada60a34380 x1875449613554432/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788571787 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788571771.995448:0:78192:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9800/0xef30c340bfb3cf7b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6640 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571771.995466:0:78192:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1788571771.995474:0:78192:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1788571771.995512:0:78192:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:0.0:1788571771.995520:0:78192:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3cf7b 00000080:00010000:0.0:1788571771.995527:0:78192:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571771.995531:0:78192:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cf7b from it ffffab0ec1cdfbf8 00010000:00010000:0.0:1788571771.995535:0:78192:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9800/0xef30c340bfb3cf7b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6640 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571771.995545:0:78192:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9800/0xef30c340bfb3cf7b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6640 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571771.995706:0:78192:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:0.0:1788571771.995741:0:78192:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571771.995763:0:78192:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571771.995775:0:78192:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9800/0xef30c340bfb3cf7b lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6640 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571771.995789:0:78192:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9800/0xef30c340bfb3cf7b lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6640 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571771.995797:0:78192:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788571771.995802:0:78192:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571771.995805:0:78192:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3cf7b 00000080:00010000:0.0:1788571771.995809:0:78192:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cf7b from it ffffab0ec1cdfb38 00010000:00010000:0.0:1788571771.995813:0:78192:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9800/0xef30c340bfb3cf7b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6640 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571771.995822:0:78192:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9800/0xef30c340bfb3cf7b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6640 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788571771.995831:0:78192:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:0.0:1788571771.995837:0:78192:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33a.replay-single--3, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), file ffff9ada600a7980, open_flags 8042, mode 81a4 opened 0 00000080:00200000:0.0:1788571771.995848:0:78192:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33a.replay-single--3, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000002:00010000:0.0:1788571771.995908:0:78192:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33a.replay-single--3,[0x200015bc1:0x4:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:0.0:1788571771.995922:0:78192:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9800/0xef30c340bfb3cf7b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b82f6640 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571771.995981:0:78192:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9800/0xef30c340bfb3cf7b lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f6640 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571771.995989:0:78192:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1788571771.995993:0:78192:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9800/0xef30c340bfb3cf7b lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f6640 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571771.996036:0:78192:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3cf82 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78192 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571771.996044:0:78192:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3cf82 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78192 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:0.0:1788571771.996058:0:78192:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:0.0:1788571771.996067:0:78192:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada516f3480 x1875449613554688/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:0.0:1788571771.996078:0:78192:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3cf82 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78192 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571771.996106:0:78192:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f3480 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78192:1875449613554688:192.168.202.113@tcp:101:createmany.0 00000100:00100000:0.0:1788571771.996298:0:78192:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada465f4000 going to sleep for 11 seconds 00000100:00100000:0.0:1788571772.003750:0:78192:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f3480 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78192:1875449613554688:192.168.202.113@tcp:101:createmany.0 00010000:00010000:0.0:1788571772.003795:0:78192:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3cf82 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f665c expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.003834:0:78192:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3cf82 lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x4:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f665c expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.003844:0:78192:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3cf82 lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f665c expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.003910:0:78192:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3cf82 lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f665c expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.003926:0:78192:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3cf82 lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f665c expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.003945:0:78192:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3cf82 lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f665c expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.003955:0:78192:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3cf82 lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f665c expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.003965:0:78192:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3cf82 lrc: 4/1,0 mode: PR/PR res: [0x200015bc1:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f665c expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788571772.003985:0:78192:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9ada516f3480 x1875449613554688/t193273528327(193273528327) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/664 e 0 to 0 dl 1788571787 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:0.0:1788571772.004001:0:78192:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada516f3480 x1875449613554688/t193273528327(193273528327) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/664 e 0 to 0 dl 1788571787 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788571772.004017:0:78192:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3cf82 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f665c expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:0.0:1788571772.004035:0:78192:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3cf82 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f665c expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.004051:0:78192:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8590023617/4 (0 0) 00000080:00200000:0.0:1788571772.004060:0:78192:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1788571772.004078:0:78192:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33a.replay-single--3, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000080:00200000:0.0:1788571772.004118:0:78192:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200015bc1:0x4:0x0](ffff9ada5825cb08) 00000080:00200000:0.0:1788571772.004222:0:78192:0:(file.c:6843:ll_layout_conf()) [0x200015bc1:0x4:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:0.0:1788571772.004229:0:78192:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200015bc1:0x4:0x0](ffff9ada5825cb08) 00000080:00010000:0.0:1788571772.004723:0:78192:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0x4:0x0](ffff9ada5825cb08) for lock 0xef30c340bfb3cf82 00000080:00010000:0.0:1788571772.004740:0:78192:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0x4:0x0](ffff9ada5825cb08) 00000080:00200008:0.0:1788571772.004762:0:78192:0:(file.c:1090:ll_file_open()) START file f33a.replay-single--3:[0x200015bc1:0x4:0x0](ffff9ada5825cb08), flags 100102 00000002:00100000:0.0:1788571772.004797:0:78192:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada516f3480 x1875449613554688/t193273528327(193273528327) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571787 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:0.0:1788571772.004843:0:78192:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cf82 from it ffff9ada517611e0 00010000:00010000:0.0:1788571772.004849:0:78192:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3cf82 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f665c expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.004907:0:78192:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3cf82 lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f665c expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571772.004941:0:78192:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200015bc1:0x4:0x0](ffff9ada5825cb08), inode mode 81a4 mask 40 00000080:00200008:0.0:1788571772.004985:0:78192:0:(file.c:479:ll_file_release()) START file f33a.replay-single--3:[0x200015bc1:0x4:0x0](ffff9ada5825cb08), flags 100002 00000002:00100000:0.0:1788571772.005057:0:78192:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9ada516f3480 x1875449613554688/t193273528327(193273528327) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571787 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:0.0:1788571772.005071:0:78192:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:0.0:1788571772.005077:0:78192:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada516f0a80 x1875449613554944/t0(0) o35->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:0.0:1788571772.005107:0:78192:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0a80 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78192:1875449613554944:192.168.202.113@tcp:35:createmany.0 00000100:00100000:0.0:1788571772.005226:0:78192:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada465f4f00 going to sleep for 11 seconds 00000100:00100000:0.0:1788571772.010191:0:78192:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0a80 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78192:1875449613554944:192.168.202.113@tcp:35:createmany.0 00000002:00100000:0.0:1788571772.010245:0:78192:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9ada516f3480 x1875449613554688/t193273528327(193273528327) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571787 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1788571772.010352:0:78192:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571772.010378:0:78192:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571772.010462:0:78192:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617000/0xef30c340bfb3cf89 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78192 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.010474:0:78192:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617000/0xef30c340bfb3cf89 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78192 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.010486:0:78192:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617000/0xef30c340bfb3cf89 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78192 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571772.010511:0:78192:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f3100 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78192:1875449613555072:192.168.202.113@tcp:101:createmany.0 00000100:00100000:0.0:1788571772.010665:0:78192:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada465f4f00 going to sleep for 11 seconds 00000100:00100000:0.0:1788571772.017301:0:78192:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f3100 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78192:1875449613555072:192.168.202.113@tcp:101:createmany.0 00010000:00010000:0.0:1788571772.017340:0:78192:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617000/0xef30c340bfb3cf89 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6671 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.017357:0:78192:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617000/0xef30c340bfb3cf89 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6671 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.017373:0:78192:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617000/0xef30c340bfb3cf89 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6671 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.017388:0:78192:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617000/0xef30c340bfb3cf89 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6671 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.017405:0:78192:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617000/0xef30c340bfb3cf89 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6671 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.017415:0:78192:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617000/0xef30c340bfb3cf89 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6671 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.017424:0:78192:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617000/0xef30c340bfb3cf89 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6671 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788571772.017437:0:78192:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada516f3100 x1875449613555072/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788571788 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788571772.017461:0:78192:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617000/0xef30c340bfb3cf89 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6671 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.017478:0:78192:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1788571772.017486:0:78192:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1788571772.017526:0:78192:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:0.0:1788571772.017535:0:78192:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3cf89 00000080:00010000:0.0:1788571772.017542:0:78192:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571772.017546:0:78192:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cf89 from it ffffab0ec1cdfbf8 00010000:00010000:0.0:1788571772.017550:0:78192:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617000/0xef30c340bfb3cf89 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6671 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.017559:0:78192:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617000/0xef30c340bfb3cf89 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6671 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571772.017593:0:78192:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:0.0:1788571772.017634:0:78192:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571772.017655:0:78192:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571772.017667:0:78192:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617000/0xef30c340bfb3cf89 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6671 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571772.017681:0:78192:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617000/0xef30c340bfb3cf89 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6671 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571772.017689:0:78192:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788571772.017695:0:78192:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571772.017698:0:78192:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3cf89 00000080:00010000:0.0:1788571772.017703:0:78192:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cf89 from it ffffab0ec1cdfb38 00010000:00010000:0.0:1788571772.017706:0:78192:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617000/0xef30c340bfb3cf89 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6671 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571772.017715:0:78192:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617000/0xef30c340bfb3cf89 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6671 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788571772.017725:0:78192:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:0.0:1788571772.017731:0:78192:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33a.replay-single--4, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), file ffff9ada6064ea80, open_flags 8042, mode 81a4 opened 0 00000080:00200000:0.0:1788571772.017741:0:78192:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33a.replay-single--4, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000002:00010000:0.0:1788571772.017768:0:78192:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33a.replay-single--4,[0x200015bc1:0x5:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:0.0:1788571772.017783:0:78192:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617000/0xef30c340bfb3cf89 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b82f6671 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571772.017850:0:78192:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617000/0xef30c340bfb3cf89 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f6671 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571772.018221:0:78192:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1788571772.018225:0:78192:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617000/0xef30c340bfb3cf89 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f6671 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571772.018443:0:78192:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8e00/0xef30c340bfb3cf90 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78192 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.018452:0:78192:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8e00/0xef30c340bfb3cf90 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78192 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1788571772.018468:0:78192:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1788571772.018479:0:78192:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada516f3100 x1875449613555328/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1788571772.018491:0:78192:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8e00/0xef30c340bfb3cf90 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78192 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571772.018519:0:78192:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f3100 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78192:1875449613555328:192.168.202.113@tcp:101:createmany.0 00000100:00100000:2.0:1788571772.018793:0:78192:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada4137db40 going to sleep for 11 seconds 00000100:00100000:0.0:1788571772.024722:0:78192:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f3100 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78192:1875449613555328:192.168.202.113@tcp:101:createmany.0 00010000:00010000:0.0:1788571772.024764:0:78192:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8e00/0xef30c340bfb3cf90 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f668d expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.024802:0:78192:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8e00/0xef30c340bfb3cf90 lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x5:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f668d expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.024818:0:78192:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8e00/0xef30c340bfb3cf90 lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f668d expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.024833:0:78192:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8e00/0xef30c340bfb3cf90 lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f668d expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.024847:0:78192:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8e00/0xef30c340bfb3cf90 lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f668d expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.025499:0:78192:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8e00/0xef30c340bfb3cf90 lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f668d expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.025510:0:78192:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8e00/0xef30c340bfb3cf90 lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f668d expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.025520:0:78192:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8e00/0xef30c340bfb3cf90 lrc: 4/1,0 mode: PR/PR res: [0x200015bc1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f668d expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788571772.025536:0:78192:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9ada516f3100 x1875449613555328/t193273528329(193273528329) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/664 e 0 to 0 dl 1788571788 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:0.0:1788571772.025553:0:78192:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada516f3100 x1875449613555328/t193273528329(193273528329) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/664 e 0 to 0 dl 1788571788 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788571772.025571:0:78192:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8e00/0xef30c340bfb3cf90 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f668d expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1788571772.027455:0:78192:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8e00/0xef30c340bfb3cf90 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f668d expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.027491:0:78192:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8590023617/5 (0 0) 00000080:00200000:2.0:1788571772.027498:0:78192:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1788571772.027518:0:78192:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33a.replay-single--4, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000080:00200000:2.0:1788571772.027562:0:78192:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200015bc1:0x5:0x0](ffff9ada58193248) 00000080:00200000:2.0:1788571772.027732:0:78192:0:(file.c:6843:ll_layout_conf()) [0x200015bc1:0x5:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1788571772.027738:0:78192:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200015bc1:0x5:0x0](ffff9ada58193248) 00000080:00010000:2.0:1788571772.027743:0:78192:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0x5:0x0](ffff9ada58193248) for lock 0xef30c340bfb3cf90 00000080:00010000:2.0:1788571772.027757:0:78192:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0x5:0x0](ffff9ada58193248) 00000080:00200008:2.0:1788571772.027782:0:78192:0:(file.c:1090:ll_file_open()) START file f33a.replay-single--4:[0x200015bc1:0x5:0x0](ffff9ada58193248), flags 100102 00000002:00100000:2.0:1788571772.027820:0:78192:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada516f3100 x1875449613555328/t193273528329(193273528329) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571788 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1788571772.028015:0:78192:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cf90 from it ffff9ada517611e0 00010000:00010000:2.0:1788571772.028023:0:78192:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8e00/0xef30c340bfb3cf90 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f668d expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.028037:0:78192:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8e00/0xef30c340bfb3cf90 lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f668d expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571772.028071:0:78192:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200015bc1:0x5:0x0](ffff9ada58193248), inode mode 81a4 mask 40 00000080:00200008:2.0:1788571772.028112:0:78192:0:(file.c:479:ll_file_release()) START file f33a.replay-single--4:[0x200015bc1:0x5:0x0](ffff9ada58193248), flags 100002 00000002:00100000:2.0:1788571772.028172:0:78192:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9ada516f3100 x1875449613555328/t193273528329(193273528329) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571788 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1788571772.028182:0:78192:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1788571772.028186:0:78192:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada60a36d80 x1875449613555584/t0(0) o35->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1788571772.028209:0:78192:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a36d80 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78192:1875449613555584:192.168.202.113@tcp:35:createmany.0 00000100:00100000:2.0:1788571772.028315:0:78192:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada4137d6c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788571772.030662:0:78192:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a36d80 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78192:1875449613555584:192.168.202.113@tcp:35:createmany.0 00000002:00100000:0.0:1788571772.030710:0:78192:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9ada516f3100 x1875449613555328/t193273528329(193273528329) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571788 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1788571772.030970:0:78192:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571772.031001:0:78192:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571772.031075:0:78192:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617000/0xef30c340bfb3cf97 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78192 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.031086:0:78192:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617000/0xef30c340bfb3cf97 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78192 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.031099:0:78192:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617000/0xef30c340bfb3cf97 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78192 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571772.031120:0:78192:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f3b80 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78192:1875449613555712:192.168.202.113@tcp:101:createmany.0 00000080:00200000:0.0:1788571772.105364:0:78192:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571772.105395:0:78192:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571772.105477:0:78192:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3cfcf lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78192 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.105489:0:78192:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3cfcf lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78192 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.105500:0:78192:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3cfcf lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78192 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571772.105527:0:78192:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f2a00 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78192:1875449613558272:192.168.202.113@tcp:101:createmany.0 00000100:00100000:0.0:1788571772.105697:0:78192:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada465f49c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788571772.109169:0:78192:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f2a00 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78192:1875449613558272:192.168.202.113@tcp:101:createmany.0 00010000:00010000:0.0:1788571772.109203:0:78192:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3cfcf lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6766 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.109217:0:78192:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3cfcf lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6766 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.109324:0:78192:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3cfcf lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6766 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.109337:0:78192:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3cfcf lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6766 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.109353:0:78192:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3cfcf lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6766 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.109363:0:78192:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3cfcf lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6766 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.109371:0:78192:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3cfcf lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6766 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788571772.109384:0:78192:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada516f2a00 x1875449613558272/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788571788 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788571772.109399:0:78192:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3cfcf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6766 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.109414:0:78192:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1788571772.109421:0:78192:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1788571772.109457:0:78192:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:0.0:1788571772.109463:0:78192:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3cfcf 00000080:00010000:0.0:1788571772.109468:0:78192:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571772.109472:0:78192:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cfcf from it ffffab0ec1cdfbf8 00010000:00010000:0.0:1788571772.109476:0:78192:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3cfcf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6766 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.109486:0:78192:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3cfcf lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6766 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571772.109515:0:78192:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:0.0:1788571772.109545:0:78192:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571772.109609:0:78192:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571772.109628:0:78192:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3cfcf lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6766 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571772.109639:0:78192:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3cfcf lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6766 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571772.109647:0:78192:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788571772.109652:0:78192:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571772.109655:0:78192:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3cfcf 00000080:00010000:0.0:1788571772.109659:0:78192:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cfcf from it ffffab0ec1cdfb38 00010000:00010000:0.0:1788571772.109662:0:78192:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3cfcf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6766 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571772.109670:0:78192:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3cfcf lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6766 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788571772.109683:0:78192:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:0.0:1788571772.109689:0:78192:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33a.replay-single--9, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), file ffff9ada6064ea80, open_flags 8042, mode 81a4 opened 0 00000080:00200000:0.0:1788571772.109699:0:78192:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33a.replay-single--9, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000002:00010000:0.0:1788571772.109722:0:78192:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33a.replay-single--9,[0x200015bc1:0xa:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:0.0:1788571772.109735:0:78192:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3cfcf lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b82f6766 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571772.109789:0:78192:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3cfcf lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f6766 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571772.109797:0:78192:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1788571772.110011:0:78192:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3cfcf lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f6766 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571772.110055:0:78192:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616400/0xef30c340bfb3cfd6 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78192 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.110062:0:78192:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616400/0xef30c340bfb3cfd6 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78192 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:0.0:1788571772.110075:0:78192:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:0.0:1788571772.110083:0:78192:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada516f2a00 x1875449613558528/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:0.0:1788571772.110094:0:78192:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616400/0xef30c340bfb3cfd6 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78192 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571772.110118:0:78192:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f2a00 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78192:1875449613558528:192.168.202.113@tcp:101:createmany.0 00000100:00100000:0.0:1788571772.110420:0:78192:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada465f49c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788571772.129407:0:78192:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f2a00 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78192:1875449613558528:192.168.202.113@tcp:101:createmany.0 00010000:00010000:0.0:1788571772.129445:0:78192:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616400/0xef30c340bfb3cfd6 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6782 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.129487:0:78192:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616400/0xef30c340bfb3cfd6 lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6782 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.129498:0:78192:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616400/0xef30c340bfb3cfd6 lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6782 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.129670:0:78192:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616400/0xef30c340bfb3cfd6 lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6782 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.129687:0:78192:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616400/0xef30c340bfb3cfd6 lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6782 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.129708:0:78192:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616400/0xef30c340bfb3cfd6 lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6782 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.129718:0:78192:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616400/0xef30c340bfb3cfd6 lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6782 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.129726:0:78192:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616400/0xef30c340bfb3cfd6 lrc: 4/1,0 mode: PR/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6782 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788571772.129742:0:78192:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9ada516f2a00 x1875449613558528/t193273528339(193273528339) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/664 e 0 to 0 dl 1788571788 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:0.0:1788571772.129758:0:78192:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada516f2a00 x1875449613558528/t193273528339(193273528339) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/664 e 0 to 0 dl 1788571788 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788571772.129776:0:78192:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616400/0xef30c340bfb3cfd6 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6782 expref: -99 pid: 78192 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:0.0:1788571772.129793:0:78192:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616400/0xef30c340bfb3cfd6 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6782 expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.129807:0:78192:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8590023617/10 (0 0) 00000080:00200000:0.0:1788571772.129815:0:78192:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1788571772.129832:0:78192:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33a.replay-single--9, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000080:00200000:0.0:1788571772.129924:0:78192:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200015bc1:0xa:0x0](ffff9ada5825f448) 00000080:00200000:0.0:1788571772.130034:0:78192:0:(file.c:6843:ll_layout_conf()) [0x200015bc1:0xa:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:0.0:1788571772.130040:0:78192:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200015bc1:0xa:0x0](ffff9ada5825f448) 00000080:00010000:0.0:1788571772.130044:0:78192:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0xa:0x0](ffff9ada5825f448) for lock 0xef30c340bfb3cfd6 00000080:00010000:0.0:1788571772.130058:0:78192:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0xa:0x0](ffff9ada5825f448) 00000080:00200008:0.0:1788571772.130078:0:78192:0:(file.c:1090:ll_file_open()) START file f33a.replay-single--9:[0x200015bc1:0xa:0x0](ffff9ada5825f448), flags 100102 00000002:00100000:0.0:1788571772.130104:0:78192:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada516f2a00 x1875449613558528/t193273528339(193273528339) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571788 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:0.0:1788571772.131066:0:78192:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cfd6 from it ffff9ada51761240 00010000:00010000:0.0:1788571772.131082:0:78192:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616400/0xef30c340bfb3cfd6 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f6782 expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.131097:0:78192:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616400/0xef30c340bfb3cfd6 lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f6782 expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571772.131134:0:78192:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200015bc1:0xa:0x0](ffff9ada5825f448), inode mode 81a4 mask 40 00000080:00200008:0.0:1788571772.131408:0:78192:0:(file.c:479:ll_file_release()) START file f33a.replay-single--9:[0x200015bc1:0xa:0x0](ffff9ada5825f448), flags 100002 00000002:00100000:0.0:1788571772.131921:0:78192:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9ada516f2a00 x1875449613558528/t193273528339(193273528339) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571788 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:0.0:1788571772.131939:0:78192:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:0.0:1788571772.131947:0:78192:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada516f0e00 x1875449613558784/t0(0) o35->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:0.0:1788571772.131979:0:78192:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0e00 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78192:1875449613558784:192.168.202.113@tcp:35:createmany.0 00000100:00100000:0.0:1788571772.132169:0:78192:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada465f4d80 going to sleep for 11 seconds 00000100:00100000:0.0:1788571772.134533:0:78192:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0e00 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78192:1875449613558784:192.168.202.113@tcp:35:createmany.0 00000002:00100000:0.0:1788571772.134785:0:78192:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9ada516f2a00 x1875449613558528/t193273528339(193273528339) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571788 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1788571772.138743:0:77477:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571772.138785:0:77477:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571772.139064:0:77477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cfdd lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 77477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.139078:0:77477:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cfdd lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.139090:0:77477:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cfdd lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 77477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571772.139123:0:77477:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a34a80 pname:cluuid:pid:xid:nid:opc:job bash:35eb2d60-2de0-4b00-9815-d5153edd0057:77477:1875449613558912:192.168.202.113@tcp:101:bash.0 00000100:00100000:2.0:1788571772.139233:0:77477:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada4137df00 going to sleep for 11 seconds 00000100:00100000:0.0:1788571772.147116:0:77477:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a34a80 pname:cluuid:pid:xid:nid:opc:job bash:35eb2d60-2de0-4b00-9815-d5153edd0057:77477:1875449613558912:192.168.202.113@tcp:101:bash.0 00010000:00010000:0.0:1788571772.147156:0:77477:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cfdd lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6797 expref: -99 pid: 77477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.147170:0:77477:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cfdd lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6797 expref: -99 pid: 77477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.147184:0:77477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cfdd lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6797 expref: -99 pid: 77477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.147197:0:77477:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cfdd lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6797 expref: -99 pid: 77477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.147213:0:77477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cfdd lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6797 expref: -99 pid: 77477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.147223:0:77477:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cfdd lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6797 expref: -99 pid: 77477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.147232:0:77477:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cfdd lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6797 expref: -99 pid: 77477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788571772.147261:0:77477:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada60a34a80 x1875449613558912/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788571788 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788571772.147279:0:77477:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cfdd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6797 expref: -99 pid: 77477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.147294:0:77477:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1788571772.147302:0:77477:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1788571772.147339:0:77477:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:0.0:1788571772.147347:0:77477:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3cfdd 00000080:00010000:0.0:1788571772.147353:0:77477:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571772.147357:0:77477:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cfdd from it ffffab0ec16d3bb0 00010000:00010000:0.0:1788571772.147360:0:77477:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cfdd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6797 expref: -99 pid: 77477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.147370:0:77477:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cfdd lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6797 expref: -99 pid: 77477 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571772.147403:0:77477:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:0.0:1788571772.147420:0:77477:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 304000 00000080:00200008:0.0:1788571772.147450:0:77477:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000002:00010000:0.0:1788571772.147478:0:77477:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: readdir flags 0 00010000:00010000:0.0:1788571772.147488:0:77477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cfdd lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6797 expref: -99 pid: 77477 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571772.147500:0:77477:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cfdd lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6797 expref: -99 pid: 77477 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000100:00100000:0.0:1788571772.148235:0:77477:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f2680 pname:cluuid:pid:xid:nid:opc:job bash:35eb2d60-2de0-4b00-9815-d5153edd0057:77477:1875449613559040:192.168.202.113@tcp:37:bash.0 00000100:00100000:0.0:1788571772.148356:0:77477:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada465f4d80 going to sleep for 11 seconds 00000080:00200000:2.0:1788571772.173157:0:78193:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571772.173199:0:78193:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571772.173224:0:78193:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cfdd lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6797 expref: -99 pid: 77477 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1788571772.173347:0:78193:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cfdd lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6797 expref: -99 pid: 77477 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1788571772.173359:0:78193:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571772.173371:0:78193:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571772.173375:0:78193:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3cfdd 00000080:00010000:2.0:1788571772.173384:0:78193:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cfdd from it ffffab0ec1cefb58 00010000:00010000:2.0:1788571772.173389:0:78193:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cfdd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6797 expref: -99 pid: 77477 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1788571772.173400:0:78193:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cfdd lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6797 expref: -99 pid: 77477 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:2.0:1788571772.173426:0:78193:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:2.0:1788571772.173436:0:78193:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33a.replay-single--0, flags=8 00000080:00200008:2.0:1788571772.173445:0:78193:0:(file.c:6296:ll_getattr_dentry()) START file f33a.replay-single--0:[0x200015bc1:0x1:0x0](ffff9ada582500c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1788571772.173451:0:78193:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200015bc1:0x1:0x0](ffff9ada582500c8),name=f33a.replay-single--0 00000002:00010000:2.0:1788571772.173466:0:78193:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200015bc1:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571772.173476:0:78193:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617a00/0xef30c340bfb3cf58 lrc: 2/1,0 mode: PR/PR res: [0x200015bc1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f65c9 expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571772.173485:0:78193:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617a00/0xef30c340bfb3cf58 lrc: 2/1,0 mode: PR/PR res: [0x200015bc1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f65c9 expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571772.173492:0:78193:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571772.173497:0:78193:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0x1:0x0](ffff9ada582500c8) 00000080:00010000:2.0:1788571772.173499:0:78193:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0x1:0x0](ffff9ada582500c8) for lock 0xef30c340bfb3cf58 00000080:00010000:2.0:1788571772.173503:0:78193:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cf58 from it ffffab0ec1cefcb8 00010000:00010000:2.0:1788571772.173507:0:78193:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617a00/0xef30c340bfb3cf58 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f65c9 expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571772.173515:0:78193:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617a00/0xef30c340bfb3cf58 lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f65c9 expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571772.173545:0:78193:0:(vvp_io.c:1826:vvp_io_init()) [0x200015bc1:0x1:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:2.0:1788571772.173674:0:78193:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x200015bc1:0x1:0x0] 00000008:00010000:2.0:1788571772.173704:0:78193:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a 00000008:00010000:2.0:1788571772.173708:0:78193:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff9ada512cdc30, osc lock ffff9ada4880fdb0, flags 0x400001002 00010000:00010000:2.0:1788571772.173780:0:78193:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 00000000348f9350/0xef30c340bfb3cfe4 lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x5e2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 78193 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.173792:0:78193:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x400001000 ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 00000000348f9350/0xef30c340bfb3cfe4 lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x5e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 78193 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:2.0:1788571772.174082:0:78193:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000080:00200000:2.0:1788571772.180379:0:78193:0:(file.c:1775:ll_merge_attr_nolock()) [0x200015bc1:0x1:0x0] updating i_size 0 i_blocks 0 00000020:00010000:2.0:1788571772.180388:0:78193:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9ada60bc39d0 (ffff9ada60b0d248/0) at cl_lock_release():245 00000020:00010000:2.0:1788571772.180392:0:78193:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9ada60bc39d0 (ffff9ada60b0d248/0) at cl_lock_cancel():142 00000020:00010000:2.0:1788571772.180398:0:78193:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9ada512cdc30 (ffff9ada60b0d098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1788571772.180404:0:78193:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9ada60bc39d0 (ffff9ada60b0d248/0) at cl_lock_fini():72 00000020:00010000:2.0:1788571772.180407:0:78193:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9ada512cdc30 (ffff9ada60b0d248/1) at cl_lock_fini():72 00000080:00200000:2.0:1788571772.180426:0:78193:0:(vvp_io.c:304:vvp_io_fini()) [0x200015bc1:0x1:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1788571772.180538:0:78193:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571772.180632:0:78193:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571772.180654:0:78193:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cfdd lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6797 expref: -99 pid: 77477 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:2.0:1788571772.180669:0:78193:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cfdd lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6797 expref: -99 pid: 77477 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:2.0:1788571772.180678:0:78193:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571772.180688:0:78193:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571772.180692:0:78193:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3cfdd 00000080:00010000:2.0:1788571772.180699:0:78193:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cfdd from it ffffab0ec1cefc10 00010000:00010000:2.0:1788571772.180703:0:78193:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cfdd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6797 expref: -99 pid: 77477 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:2.0:1788571772.180717:0:78193:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cfdd lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6797 expref: -99 pid: 77477 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:2.0:1788571772.180730:0:78193:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:2.0:1788571772.180744:0:78193:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33a.replay-single--0, flags=0 00000080:00200000:2.0:1788571772.180751:0:78193:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571772.180759:0:78193:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571772.180766:0:78193:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cfdd lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6797 expref: -99 pid: 77477 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:2.0:1788571772.180776:0:78193:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cfdd lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6797 expref: -99 pid: 77477 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:2.0:1788571772.180782:0:78193:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571772.180787:0:78193:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571772.180789:0:78193:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3cfdd 00000080:00010000:2.0:1788571772.180793:0:78193:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cfdd from it ffffab0ec1cefd68 00010000:00010000:2.0:1788571772.180796:0:78193:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cfdd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6797 expref: -99 pid: 77477 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:2.0:1788571772.180804:0:78193:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cfdd lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6797 expref: -99 pid: 77477 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000080:00200000:2.0:1788571772.180813:0:78193:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:2.0:1788571772.180819:0:78193:0:(namei.c:2351:ll_unlink()) VFS Op:name=f33a.replay-single--0, dir=[0x200000007:0x1:0x0](ffff9ada58125b88) 00010000:00010000:2.0:1788571772.180986:0:78193:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cfdd lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b82f6797 expref: -99 pid: 77477 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:2.0:1788571772.181069:0:78193:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617a00/0xef30c340bfb3cf58 lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0xdb7edb52b82f65c9 expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571772.181082:0:78193:0:(vvp_object.c:117:vvp_conf_set()) [0x200015bc1:0x1:0x0]: losing layout lock 00010000:00010000:2.0:1788571772.181127:0:78193:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617a00/0xef30c340bfb3cf58 lrc: 1/0,0 mode: --/PR res: [0x200015bc1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0xdb7edb52b82f65c9 expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571772.181135:0:78193:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cfdd lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f6797 expref: -99 pid: 77477 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:2.0:1788571772.181141:0:78193:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:2.0:1788571772.181144:0:78193:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617a00/0xef30c340bfb3cf58 lrc: 0/0,0 mode: --/PR res: [0x200015bc1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0xdb7edb52b82f65c9 expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571772.181162:0:78193:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3cfdd lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f6797 expref: -99 pid: 77477 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000020:00100000:2.0:1788571772.181184:0:78193:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1788571772.181197:0:78193:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada60a34380 x1875449613559424/t0(0) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788571772.181229:0:78193:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a34380 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:78193:1875449613559424:192.168.202.113@tcp:36:rm.0 00000100:00100000:2.0:1788571772.181850:0:78193:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada4137dcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788571772.195895:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613559680:192.168.202.113@tcp:101:rm.0 00010000:00010000:0.0:1788571772.206701:0:2384:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000fd3161a4/0xef30c340bfb3cff2 lrc: 6/2,0 mode: --/PR res: [0x280000401:0x5e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 78193 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.206739:0:2384:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000fd3161a4/0xef30c340bfb3cff2 lrc: 6/2,0 mode: --/PR res: [0x280000401:0x5e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f67cf expref: -99 pid: 78193 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.206756:0:2384:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000fd3161a4/0xef30c340bfb3cff2 lrc: 6/2,0 mode: PR/PR res: [0x280000401:0x5e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f67cf expref: -99 pid: 78193 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.206770:0:2384:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000fd3161a4/0xef30c340bfb3cff2 lrc: 6/2,0 mode: PR/PR res: [0x280000401:0x5e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f67cf expref: -99 pid: 78193 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1788571772.206794:0:2384:0:(osc_lock.c:187:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000fd3161a4/0xef30c340bfb3cff2 lrc: 5/2,0 mode: PR/PR res: [0x280000401:0x5e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f67cf expref: -99 pid: 78193 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:0.0:1788571772.206807:0:2384:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00010000:00010000:0.0:1788571772.206827:0:2384:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000fd3161a4/0xef30c340bfb3cff2 lrc: 6/2,0 mode: PR/PR res: [0x280000401:0x5e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0xdb7edb52b82f67cf expref: -99 pid: 78193 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.206837:0:2384:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000fd3161a4/0xef30c340bfb3cff2 lrc: 5/1,0 mode: PR/PR res: [0x280000401:0x5e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0xdb7edb52b82f67cf expref: -99 pid: 78193 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.206845:0:2384:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000fd3161a4/0xef30c340bfb3cff2 lrc: 5/1,0 mode: PR/PR res: [0x280000401:0x5e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0xdb7edb52b82f67cf expref: -99 pid: 78193 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.208316:0:2384:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000fd3161a4/0xef30c340bfb3cff2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0xdb7edb52b82f67cf expref: -99 pid: 78193 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:0.0:1788571772.208347:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613559680:192.168.202.113@tcp:101:rm.0 00000080:00200000:2.0:1788571772.242662:0:78193:0:(file.c:1775:ll_merge_attr_nolock()) [0x200015bc1:0x3:0x0] updating i_size 0 i_blocks 0 00000020:00010000:2.0:1788571772.242671:0:78193:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9ada51684de0 (ffff9ada60b68998/0) at cl_lock_release():245 00000020:00010000:2.0:1788571772.242675:0:78193:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9ada51684de0 (ffff9ada60b68998/0) at cl_lock_cancel():142 00000020:00010000:2.0:1788571772.242682:0:78193:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9ada51899730 (ffff9ada60b68758/1) at cl_lock_cancel():142 00000020:00010000:2.0:1788571772.242688:0:78193:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9ada51684de0 (ffff9ada60b68998/0) at cl_lock_fini():72 00000020:00010000:2.0:1788571772.242691:0:78193:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9ada51899730 (ffff9ada60b68998/1) at cl_lock_fini():72 00000080:00200000:2.0:1788571772.242744:0:78193:0:(vvp_io.c:304:vvp_io_fini()) [0x200015bc1:0x3:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1788571772.242915:0:78193:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571772.242950:0:78193:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571772.242972:0:78193:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e000/0xef30c340bfb3cff9 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f67f9 expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571772.242989:0:78193:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e000/0xef30c340bfb3cff9 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f67f9 expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571772.243002:0:78193:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571772.243013:0:78193:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571772.243018:0:78193:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3cff9 00000080:00010000:2.0:1788571772.243026:0:78193:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cff9 from it ffffab0ec1cefc10 00010000:00010000:2.0:1788571772.243030:0:78193:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e000/0xef30c340bfb3cff9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f67f9 expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571772.243038:0:78193:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e000/0xef30c340bfb3cff9 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f67f9 expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571772.243053:0:78193:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00010000:00010000:0.0:1788571772.259722:0:2383:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000658e4e68/0xef30c340bfb3d00e lrc: 6/2,0 mode: --/PR res: [0x280000401:0x5e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 78193 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.259759:0:2383:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000658e4e68/0xef30c340bfb3d00e lrc: 6/2,0 mode: --/PR res: [0x280000401:0x5e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6831 expref: -99 pid: 78193 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.259786:0:2383:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000658e4e68/0xef30c340bfb3d00e lrc: 6/2,0 mode: PR/PR res: [0x280000401:0x5e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6831 expref: -99 pid: 78193 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.259802:0:2383:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000658e4e68/0xef30c340bfb3d00e lrc: 6/2,0 mode: PR/PR res: [0x280000401:0x5e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6831 expref: -99 pid: 78193 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1788571772.259822:0:2383:0:(osc_lock.c:187:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000658e4e68/0xef30c340bfb3d00e lrc: 5/2,0 mode: PR/PR res: [0x280000401:0x5e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f6831 expref: -99 pid: 78193 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:0.0:1788571772.259836:0:2383:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00010000:00010000:0.0:1788571772.259887:0:2383:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000658e4e68/0xef30c340bfb3d00e lrc: 6/2,0 mode: PR/PR res: [0x280000401:0x5e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0xdb7edb52b82f6831 expref: -99 pid: 78193 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.259896:0:2383:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000658e4e68/0xef30c340bfb3d00e lrc: 5/1,0 mode: PR/PR res: [0x280000401:0x5e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0xdb7edb52b82f6831 expref: -99 pid: 78193 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.259904:0:2383:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000658e4e68/0xef30c340bfb3d00e lrc: 5/1,0 mode: PR/PR res: [0x280000401:0x5e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0xdb7edb52b82f6831 expref: -99 pid: 78193 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.259913:0:2383:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000658e4e68/0xef30c340bfb3d00e lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0xdb7edb52b82f6831 expref: -99 pid: 78193 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:0.0:1788571772.259939:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613560704:192.168.202.113@tcp:101:rm.0 00000080:00200000:0.0:1788571772.268353:0:78193:0:(vvp_io.c:1826:vvp_io_init()) [0x200015bc1:0x4:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1788571772.268370:0:78193:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1788571772.268403:0:78193:0:(vvp_io.c:304:vvp_io_fini()) [0x200015bc1:0x4:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1788571772.268413:0:78193:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200015bc1:0x4:0x0](ffff9ada5825cb08) 00000008:00010000:0.0:1788571772.268457:0:78193:0:(osc_object.c:202:osc_object_ast_clear()) ### update lock size 0 blocks 0 [cma]time: 0 0 0 by oinfo size 0 blocks 0 [cma]time 0 0 0 ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000658e4e68/0xef30c340bfb3d00e lrc: 1/0,0 mode: PR/PR res: [0x280000401:0x5e3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0xdb7edb52b82f6831 expref: -99 pid: 78193 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571772.268717:0:78193:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571772.268746:0:78193:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571772.269128:0:78193:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d015 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78193 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.269140:0:78193:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d015 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78193 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.269152:0:78193:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d015 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78193 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571772.269179:0:78193:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f1f80 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:78193:1875449613561088:192.168.202.113@tcp:101:rm.0 00000100:00100000:0.0:1788571772.269292:0:78193:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada465f4000 going to sleep for 11 seconds 00000100:00100000:0.0:1788571772.272205:0:78193:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f1f80 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:78193:1875449613561088:192.168.202.113@tcp:101:rm.0 00010000:00010000:0.0:1788571772.272257:0:78193:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d015 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f685b expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.272272:0:78193:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d015 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f685b expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.272292:0:78193:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d015 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f685b expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.272304:0:78193:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d015 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f685b expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.272318:0:78193:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d015 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f685b expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.272326:0:78193:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d015 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f685b expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.272335:0:78193:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d015 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f685b expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788571772.272351:0:78193:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada516f1f80 x1875449613561088/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788571788 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788571772.272367:0:78193:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d015 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f685b expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.272383:0:78193:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1788571772.272391:0:78193:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1788571772.272431:0:78193:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:0.0:1788571772.272438:0:78193:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d015 00000080:00010000:0.0:1788571772.272445:0:78193:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571772.272449:0:78193:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d015 from it ffffab0ec1cefb58 00010000:00010000:0.0:1788571772.272453:0:78193:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d015 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f685b expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.272462:0:78193:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d015 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f685b expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571772.272494:0:78193:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:0.0:1788571772.272510:0:78193:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33a.replay-single--4, flags=8 00000080:00200008:0.0:1788571772.272526:0:78193:0:(file.c:6296:ll_getattr_dentry()) START file f33a.replay-single--4:[0x200015bc1:0x5:0x0](ffff9ada58193248), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1788571772.272537:0:78193:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200015bc1:0x5:0x0](ffff9ada58193248),name=f33a.replay-single--4 00000002:00010000:0.0:1788571772.272577:0:78193:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200015bc1:0x5:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571772.272590:0:78193:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8e00/0xef30c340bfb3cf90 lrc: 2/1,0 mode: PR/PR res: [0x200015bc1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f668d expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571772.272602:0:78193:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8e00/0xef30c340bfb3cf90 lrc: 2/1,0 mode: PR/PR res: [0x200015bc1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f668d expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571772.272609:0:78193:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788571772.272624:0:78193:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0x5:0x0](ffff9ada58193248) 00000080:00010000:0.0:1788571772.272627:0:78193:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0x5:0x0](ffff9ada58193248) for lock 0xef30c340bfb3cf90 00000080:00010000:0.0:1788571772.272631:0:78193:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cf90 from it ffffab0ec1cefcb8 00010000:00010000:0.0:1788571772.272634:0:78193:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8e00/0xef30c340bfb3cf90 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f668d expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571772.272641:0:78193:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8e00/0xef30c340bfb3cf90 lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f668d expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788571772.272667:0:78193:0:(vvp_io.c:1826:vvp_io_init()) [0x200015bc1:0x5:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:0.0:1788571772.272681:0:78193:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x200015bc1:0x5:0x0] 00000008:00010000:0.0:1788571772.272709:0:78193:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a 00000008:00010000:0.0:1788571772.272713:0:78193:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff9ada489952b0, osc lock ffff9ada7d07b8f0, flags 0x400001002 00010000:00010000:0.0:1788571772.272770:0:78193:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 00000000424f9e68/0xef30c340bfb3d01c lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x5e4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 78193 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.272781:0:78193:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x400001000 ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 00000000424f9e68/0xef30c340bfb3d01c lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x5e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 78193 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:0.0:1788571772.272820:0:78193:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000100:00100000:0.0:1788571772.272844:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613561216:192.168.202.113@tcp:101:rm.0 00010000:00010000:0.0:1788571772.277723:0:2383:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 00000000424f9e68/0xef30c340bfb3d01c lrc: 6/2,0 mode: --/PR res: [0x2c0000401:0x5e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 78193 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.277770:0:2383:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 00000000424f9e68/0xef30c340bfb3d01c lrc: 6/2,0 mode: --/PR res: [0x2c0000401:0x5e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6862 expref: -99 pid: 78193 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.277793:0:2383:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 00000000424f9e68/0xef30c340bfb3d01c lrc: 6/2,0 mode: PR/PR res: [0x2c0000401:0x5e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6862 expref: -99 pid: 78193 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.277811:0:2383:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 00000000424f9e68/0xef30c340bfb3d01c lrc: 6/2,0 mode: PR/PR res: [0x2c0000401:0x5e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6862 expref: -99 pid: 78193 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1788571772.277832:0:2383:0:(osc_lock.c:187:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 00000000424f9e68/0xef30c340bfb3d01c lrc: 5/2,0 mode: PR/PR res: [0x2c0000401:0x5e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f6862 expref: -99 pid: 78193 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:0.0:1788571772.277847:0:2383:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00010000:00010000:0.0:1788571772.278023:0:2383:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 00000000424f9e68/0xef30c340bfb3d01c lrc: 6/2,0 mode: PR/PR res: [0x2c0000401:0x5e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0xdb7edb52b82f6862 expref: -99 pid: 78193 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.278034:0:2383:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 00000000424f9e68/0xef30c340bfb3d01c lrc: 5/1,0 mode: PR/PR res: [0x2c0000401:0x5e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0xdb7edb52b82f6862 expref: -99 pid: 78193 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.278044:0:2383:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 00000000424f9e68/0xef30c340bfb3d01c lrc: 5/1,0 mode: PR/PR res: [0x2c0000401:0x5e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0xdb7edb52b82f6862 expref: -99 pid: 78193 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.278053:0:2383:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 00000000424f9e68/0xef30c340bfb3d01c lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x5e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0xdb7edb52b82f6862 expref: -99 pid: 78193 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:0.0:1788571772.278080:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613561216:192.168.202.113@tcp:101:rm.0 00000080:00200000:0.0:1788571772.292481:0:78193:0:(file.c:1775:ll_merge_attr_nolock()) [0x200015bc1:0x6:0x0] updating i_size 0 i_blocks 0 00000020:00010000:0.0:1788571772.292489:0:78193:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9ada60bc06f0 (ffff9ada60b0d2d8/0) at cl_lock_release():245 00000020:00010000:0.0:1788571772.292493:0:78193:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9ada60bc06f0 (ffff9ada60b0d2d8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1788571772.292498:0:78193:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9ada60a95f30 (ffff9ada60b68ea8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1788571772.292503:0:78193:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9ada60bc06f0 (ffff9ada60b0d2d8/0) at cl_lock_fini():72 00000020:00010000:0.0:1788571772.292507:0:78193:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9ada60a95f30 (ffff9ada60b0d2d8/1) at cl_lock_fini():72 00000080:00200000:0.0:1788571772.292523:0:78193:0:(vvp_io.c:304:vvp_io_fini()) [0x200015bc1:0x6:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1788571772.292580:0:78193:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571772.292609:0:78193:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571772.292639:0:78193:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759200/0xef30c340bfb3d023 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f688c expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571772.292761:0:78193:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759200/0xef30c340bfb3d023 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f688c expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571772.292772:0:78193:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788571772.292783:0:78193:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571772.292787:0:78193:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d023 00000080:00010000:0.0:1788571772.292794:0:78193:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d023 from it ffffab0ec1cefc10 00010000:00010000:0.0:1788571772.293031:0:78193:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759200/0xef30c340bfb3d023 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f688c expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571772.293043:0:78193:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759200/0xef30c340bfb3d023 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f688c expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788571772.293056:0:78193:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:0.0:1788571772.293070:0:78193:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33a.replay-single--5, flags=0 00000080:00200000:0.0:1788571772.293080:0:78193:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571772.293091:0:78193:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571772.293098:0:78193:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759200/0xef30c340bfb3d023 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f688c expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571772.293106:0:78193:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759200/0xef30c340bfb3d023 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f688c expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788571772.293114:0:78193:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788571772.293118:0:78193:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571772.293121:0:78193:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d023 00000080:00010000:0.0:1788571772.293124:0:78193:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d023 from it ffffab0ec1cefd68 00010000:00010000:0.0:1788571772.293127:0:78193:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759200/0xef30c340bfb3d023 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f688c expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571772.293141:0:78193:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759200/0xef30c340bfb3d023 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f688c expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1788571772.293149:0:78193:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:0.0:1788571772.293156:0:78193:0:(namei.c:2351:ll_unlink()) VFS Op:name=f33a.replay-single--5, dir=[0x200000007:0x1:0x0](ffff9ada58125b88) 00010000:00010000:0.0:1788571772.293193:0:78193:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759200/0xef30c340bfb3d023 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b82f688c expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1788571772.293224:0:78193:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163fc00/0xef30c340bfb3cf9e lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0xdb7edb52b82f66be expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788571772.293304:0:78193:0:(vvp_object.c:117:vvp_conf_set()) [0x200015bc1:0x6:0x0]: losing layout lock 00010000:00010000:0.0:1788571772.293360:0:78193:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163fc00/0xef30c340bfb3cf9e lrc: 1/0,0 mode: --/PR res: [0x200015bc1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0xdb7edb52b82f66be expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571772.293370:0:78193:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759200/0xef30c340bfb3d023 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f688c expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1788571772.293376:0:78193:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:0.0:1788571772.293379:0:78193:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163fc00/0xef30c340bfb3cf9e lrc: 0/0,0 mode: --/PR res: [0x200015bc1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0xdb7edb52b82f66be expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571772.293601:0:78193:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759200/0xef30c340bfb3d023 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f688c expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:0.0:1788571772.293637:0:78193:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1788571772.293712:0:78193:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada516f1f80 x1875449613561984/t0(0) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788571772.293749:0:78193:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f1f80 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:78193:1875449613561984:192.168.202.113@tcp:36:rm.0 00000100:00100000:0.0:1788571772.294091:0:78193:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada465f4000 going to sleep for 11 seconds 00000100:00100000:0.0:1788571772.298229:0:78193:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f1f80 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:78193:1875449613561984:192.168.202.113@tcp:36:rm.0 00000080:00200000:0.0:1788571772.298322:0:78193:0:(vvp_io.c:1826:vvp_io_init()) [0x200015bc1:0x6:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1788571772.298340:0:78193:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1788571772.298381:0:78193:0:(vvp_io.c:304:vvp_io_fini()) [0x200015bc1:0x6:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1788571772.298391:0:78193:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200015bc1:0x6:0x0](ffff9ada58255348) 00000008:00010000:0.0:1788571772.298438:0:78193:0:(osc_object.c:202:osc_object_ast_clear()) ### update lock size 0 blocks 0 [cma]time: 0 0 0 by oinfo size 0 blocks 0 [cma]time 0 0 0 ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000b4cf5d29/0xef30c340bfb3d02a lrc: 1/0,0 mode: PR/PR res: [0x280000401:0x5e4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0xdb7edb52b82f6893 expref: -99 pid: 78193 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571772.298540:0:78193:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571772.298572:0:78193:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571772.298778:0:78193:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d031 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78193 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.298788:0:78193:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d031 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78193 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.298957:0:78193:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d031 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78193 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571772.298986:0:78193:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0000 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:78193:1875449613562112:192.168.202.113@tcp:101:rm.0 00000100:00100000:0.0:1788571772.299083:0:78193:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada465f4000 going to sleep for 11 seconds 00000100:00100000:0.0:1788571772.302318:0:78193:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0000 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:78193:1875449613562112:192.168.202.113@tcp:101:rm.0 00010000:00010000:0.0:1788571772.302353:0:78193:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d031 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f68bd expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.302369:0:78193:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d031 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f68bd expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.302383:0:78193:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d031 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f68bd expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.302395:0:78193:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d031 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f68bd expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.302411:0:78193:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d031 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f68bd expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.302421:0:78193:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d031 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f68bd expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.302429:0:78193:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d031 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f68bd expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788571772.302441:0:78193:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada516f0000 x1875449613562112/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788571788 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788571772.302457:0:78193:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d031 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f68bd expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.302472:0:78193:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1788571772.302479:0:78193:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1788571772.302514:0:78193:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:0.0:1788571772.302521:0:78193:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d031 00000080:00010000:0.0:1788571772.302527:0:78193:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571772.302530:0:78193:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d031 from it ffffab0ec1cefb58 00010000:00010000:0.0:1788571772.302534:0:78193:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d031 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f68bd expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.302544:0:78193:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d031 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f68bd expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571772.302571:0:78193:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:0.0:1788571772.302586:0:78193:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33a.replay-single--6, flags=8 00000080:00200008:0.0:1788571772.302602:0:78193:0:(file.c:6296:ll_getattr_dentry()) START file f33a.replay-single--6:[0x200015bc1:0x7:0x0](ffff9ada58250908), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1788571772.303272:0:78193:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200015bc1:0x7:0x0](ffff9ada58250908),name=f33a.replay-single--6 00000002:00010000:0.0:1788571772.303307:0:78193:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200015bc1:0x7:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571772.303323:0:78193:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e800/0xef30c340bfb3cfac lrc: 2/1,0 mode: PR/PR res: [0x200015bc1:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f66ef expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571772.303337:0:78193:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e800/0xef30c340bfb3cfac lrc: 2/1,0 mode: PR/PR res: [0x200015bc1:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f66ef expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571772.303344:0:78193:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788571772.303349:0:78193:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0x7:0x0](ffff9ada58250908) 00000080:00010000:0.0:1788571772.303353:0:78193:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0x7:0x0](ffff9ada58250908) for lock 0xef30c340bfb3cfac 00000080:00010000:0.0:1788571772.303358:0:78193:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cfac from it ffffab0ec1cefcb8 00010000:00010000:0.0:1788571772.303361:0:78193:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e800/0xef30c340bfb3cfac lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f66ef expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571772.303368:0:78193:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e800/0xef30c340bfb3cfac lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f66ef expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788571772.303403:0:78193:0:(vvp_io.c:1826:vvp_io_init()) [0x200015bc1:0x7:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:0.0:1788571772.303417:0:78193:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x200015bc1:0x7:0x0] 00000008:00010000:0.0:1788571772.303444:0:78193:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a 00000008:00010000:0.0:1788571772.303450:0:78193:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff9ada48995130, osc lock ffff9ada7d07abe0, flags 0x400001002 00010000:00010000:0.0:1788571772.303520:0:78193:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 0000000005bd4a7e/0xef30c340bfb3d038 lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x5e5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 78193 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.303532:0:78193:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x400001000 ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 0000000005bd4a7e/0xef30c340bfb3d038 lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x5e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 78193 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:0.0:1788571772.303585:0:78193:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000100:00100000:0.0:1788571772.303600:0:2383:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1788571772.303639:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613562240:192.168.202.113@tcp:101:rm.0 00010000:00010000:2.0:1788571772.304223:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f200/0xef30c340bfb3cfba lrc: 2/1,0 mode: PR/PR res: [0x200015bc1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f6720 expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571772.304243:0:78194:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f200/0xef30c340bfb3cfba lrc: 2/1,0 mode: PR/PR res: [0x200015bc1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f6720 expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00010000:2.0:1788571772.304257:0:78194:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cfba from it ffffab0ec203fd78 00010000:00010000:2.0:1788571772.304261:0:78194:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f200/0xef30c340bfb3cfba lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f6720 expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571772.304270:0:78194:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f200/0xef30c340bfb3cfba lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f6720 expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571772.304750:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e400/0xef30c340bfb3cfc8 lrc: 2/1,0 mode: PR/PR res: [0x200015bc1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f6751 expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571772.304761:0:78194:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e400/0xef30c340bfb3cfc8 lrc: 2/1,0 mode: PR/PR res: [0x200015bc1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f6751 expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00010000:2.0:1788571772.304768:0:78194:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cfc8 from it ffffab0ec203fd78 00010000:00010000:2.0:1788571772.304772:0:78194:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e400/0xef30c340bfb3cfc8 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f6751 expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571772.304779:0:78194:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e400/0xef30c340bfb3cfc8 lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f6751 expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571772.305469:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616400/0xef30c340bfb3cfd6 lrc: 2/1,0 mode: PR/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f6782 expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571772.305480:0:78194:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616400/0xef30c340bfb3cfd6 lrc: 2/1,0 mode: PR/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f6782 expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00010000:2.0:1788571772.305488:0:78194:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cfd6 from it ffffab0ec203fd78 00010000:00010000:2.0:1788571772.305492:0:78194:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616400/0xef30c340bfb3cfd6 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f6782 expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571772.305500:0:78194:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616400/0xef30c340bfb3cfd6 lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f6782 expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571772.305790:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8400/0xef30c340bfb3d054 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.309338:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8400/0xef30c340bfb3d054 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.309462:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9800/0xef30c340bfb3d05b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.309470:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9800/0xef30c340bfb3d05b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.309491:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3d062 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.309498:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3d062 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.309519:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3d069 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.309526:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3d069 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.309543:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9e00/0xef30c340bfb3d070 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.309555:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9e00/0xef30c340bfb3d070 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.310362:0:2385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 000000002bdeef89/0xef30c340bfb3d04d lrc: 6/2,0 mode: --/PR res: [0x280000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 78195 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.310385:0:2385:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 000000002bdeef89/0xef30c340bfb3d04d lrc: 6/2,0 mode: --/PR res: [0x280000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f68d9 expref: -99 pid: 78195 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.310404:0:2385:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 000000002bdeef89/0xef30c340bfb3d04d lrc: 6/2,0 mode: PR/PR res: [0x280000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f68d9 expref: -99 pid: 78195 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.310419:0:2385:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 000000002bdeef89/0xef30c340bfb3d04d lrc: 6/2,0 mode: PR/PR res: [0x280000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f68d9 expref: -99 pid: 78195 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1788571772.310439:0:2385:0:(osc_lock.c:187:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 000000002bdeef89/0xef30c340bfb3d04d lrc: 5/2,0 mode: PR/PR res: [0x280000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f68d9 expref: -99 pid: 78195 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.310457:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 000000002bdeef89/0xef30c340bfb3d04d lrc: 5/2,0 mode: PR/PR res: [0x280000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f68d9 expref: -99 pid: 78195 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.310467:0:2385:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 000000002bdeef89/0xef30c340bfb3d04d lrc: 4/1,0 mode: PR/PR res: [0x280000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f68d9 expref: -99 pid: 78195 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.310476:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 000000002bdeef89/0xef30c340bfb3d04d lrc: 4/1,0 mode: PR/PR res: [0x280000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f68d9 expref: -99 pid: 78195 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.310487:0:2385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 000000002bdeef89/0xef30c340bfb3d04d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f68d9 expref: -99 pid: 78195 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:2.0:1788571772.310508:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a48a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613562624:192.168.202.113@tcp:101:ll_agl.0 00010000:00010000:2.0:1788571772.310529:0:2385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 000000009657ad4d/0xef30c340bfb3d046 lrc: 6/2,0 mode: --/PR res: [0x2c0000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 78195 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.310539:0:2385:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 000000009657ad4d/0xef30c340bfb3d046 lrc: 6/2,0 mode: --/PR res: [0x2c0000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f68d2 expref: -99 pid: 78195 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.310548:0:2385:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 000000009657ad4d/0xef30c340bfb3d046 lrc: 6/2,0 mode: PR/PR res: [0x2c0000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f68d2 expref: -99 pid: 78195 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.310558:0:2385:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 000000009657ad4d/0xef30c340bfb3d046 lrc: 6/2,0 mode: PR/PR res: [0x2c0000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f68d2 expref: -99 pid: 78195 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1788571772.310570:0:2385:0:(osc_lock.c:187:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 000000009657ad4d/0xef30c340bfb3d046 lrc: 5/2,0 mode: PR/PR res: [0x2c0000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f68d2 expref: -99 pid: 78195 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.310580:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 000000009657ad4d/0xef30c340bfb3d046 lrc: 5/2,0 mode: PR/PR res: [0x2c0000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f68d2 expref: -99 pid: 78195 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.310588:0:2385:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 000000009657ad4d/0xef30c340bfb3d046 lrc: 4/1,0 mode: PR/PR res: [0x2c0000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f68d2 expref: -99 pid: 78195 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.310596:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 000000009657ad4d/0xef30c340bfb3d046 lrc: 4/1,0 mode: PR/PR res: [0x2c0000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f68d2 expref: -99 pid: 78195 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.310604:0:2385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 000000009657ad4d/0xef30c340bfb3d046 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f68d2 expref: -99 pid: 78195 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:2.0:1788571772.310625:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a49500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613562496:192.168.202.113@tcp:101:ll_agl.0 00010000:00010000:2.0:1788571772.310641:0:2385:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 0000000078fe1a46/0xef30c340bfb3d03f lrc: 6/2,0 mode: --/PR res: [0x280000401:0x5e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 78195 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.310650:0:2385:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 0000000078fe1a46/0xef30c340bfb3d03f lrc: 6/2,0 mode: --/PR res: [0x280000401:0x5e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f68cb expref: -99 pid: 78195 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.310661:0:2385:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 0000000078fe1a46/0xef30c340bfb3d03f lrc: 6/2,0 mode: PR/PR res: [0x280000401:0x5e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f68cb expref: -99 pid: 78195 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.310670:0:2385:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 0000000078fe1a46/0xef30c340bfb3d03f lrc: 6/2,0 mode: PR/PR res: [0x280000401:0x5e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f68cb expref: -99 pid: 78195 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1788571772.310680:0:2385:0:(osc_lock.c:187:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 0000000078fe1a46/0xef30c340bfb3d03f lrc: 5/2,0 mode: PR/PR res: [0x280000401:0x5e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f68cb expref: -99 pid: 78195 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.310691:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 0000000078fe1a46/0xef30c340bfb3d03f lrc: 5/2,0 mode: PR/PR res: [0x280000401:0x5e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f68cb expref: -99 pid: 78195 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.310700:0:2385:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 0000000078fe1a46/0xef30c340bfb3d03f lrc: 4/1,0 mode: PR/PR res: [0x280000401:0x5e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f68cb expref: -99 pid: 78195 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.310706:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 0000000078fe1a46/0xef30c340bfb3d03f lrc: 4/1,0 mode: PR/PR res: [0x280000401:0x5e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f68cb expref: -99 pid: 78195 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.310715:0:2385:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 0000000078fe1a46/0xef30c340bfb3d03f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x5e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f68cb expref: -99 pid: 78195 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:2.0:1788571772.310726:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613562368:192.168.202.113@tcp:101:ll_agl.0 00000080:00200000:0.0:1788571772.312020:0:78193:0:(file.c:1775:ll_merge_attr_nolock()) [0x200015bc1:0x7:0x0] updating i_size 0 i_blocks 0 00000020:00010000:0.0:1788571772.312028:0:78193:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9ada60bc06f0 (ffff9ada60b0d2d8/0) at cl_lock_release():245 00000020:00010000:0.0:1788571772.312032:0:78193:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9ada60bc06f0 (ffff9ada60b0d2d8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1788571772.312039:0:78193:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9ada48995130 (ffff9ada60b68ea8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1788571772.312044:0:78193:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9ada60bc06f0 (ffff9ada60b0d2d8/0) at cl_lock_fini():72 00000020:00010000:0.0:1788571772.312046:0:78193:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9ada48995130 (ffff9ada60b0d2d8/1) at cl_lock_fini():72 00000080:00200000:0.0:1788571772.312064:0:78193:0:(vvp_io.c:304:vvp_io_fini()) [0x200015bc1:0x7:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1788571772.312128:0:78193:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571772.312158:0:78193:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571772.312178:0:78193:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d031 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 8 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f68bd expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571772.312192:0:78193:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d031 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f68bd expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571772.312202:0:78193:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788571772.312210:0:78193:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571772.312214:0:78193:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d031 00000080:00010000:0.0:1788571772.312220:0:78193:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d031 from it ffffab0ec1cefc10 00010000:00010000:0.0:1788571772.312224:0:78193:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d031 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f68bd expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571772.312232:0:78193:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d031 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f68bd expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788571772.312253:0:78193:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:0.0:1788571772.312265:0:78193:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33a.replay-single--6, flags=0 00000080:00200000:0.0:1788571772.312293:0:78193:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571772.312301:0:78193:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571772.312307:0:78193:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d031 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 8 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f68bd expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571772.312316:0:78193:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d031 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f68bd expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788571772.312323:0:78193:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788571772.312327:0:78193:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571772.312329:0:78193:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d031 00000080:00010000:0.0:1788571772.312333:0:78193:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d031 from it ffffab0ec1cefd68 00010000:00010000:0.0:1788571772.312336:0:78193:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d031 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f68bd expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571772.312350:0:78193:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d031 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f68bd expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1788571772.312358:0:78193:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:0.0:1788571772.312364:0:78193:0:(namei.c:2351:ll_unlink()) VFS Op:name=f33a.replay-single--6, dir=[0x200000007:0x1:0x0](ffff9ada58125b88) 00010000:00010000:0.0:1788571772.312397:0:78193:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d031 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 8 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b82f68bd expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1788571772.312426:0:78193:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e800/0xef30c340bfb3cfac lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0xdb7edb52b82f66ef expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788571772.312436:0:78193:0:(vvp_object.c:117:vvp_conf_set()) [0x200015bc1:0x7:0x0]: losing layout lock 00010000:00010000:0.0:1788571772.312485:0:78193:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e800/0xef30c340bfb3cfac lrc: 1/0,0 mode: --/PR res: [0x200015bc1:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0xdb7edb52b82f66ef expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571772.312493:0:78193:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d031 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f68bd expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1788571772.312499:0:78193:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:0.0:1788571772.312502:0:78193:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e800/0xef30c340bfb3cfac lrc: 0/0,0 mode: --/PR res: [0x200015bc1:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0xdb7edb52b82f66ef expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571772.312526:0:78193:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d031 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f68bd expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:0.0:1788571772.312546:0:78193:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1788571772.312557:0:78193:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada516f3800 x1875449613563008/t0(0) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788571772.312587:0:78193:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f3800 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:78193:1875449613563008:192.168.202.113@tcp:36:rm.0 00000100:00100000:0.0:1788571772.312697:0:78193:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada465f4000 going to sleep for 11 seconds 00000100:00100000:0.0:1788571772.320050:0:78193:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f3800 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:78193:1875449613563008:192.168.202.113@tcp:36:rm.0 00000080:00200000:0.0:1788571772.320139:0:78193:0:(vvp_io.c:1826:vvp_io_init()) [0x200015bc1:0x7:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1788571772.320157:0:78193:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1788571772.320196:0:78193:0:(vvp_io.c:304:vvp_io_fini()) [0x200015bc1:0x7:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1788571772.320207:0:78193:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200015bc1:0x7:0x0](ffff9ada58250908) 00000008:00010000:0.0:1788571772.321098:0:78193:0:(osc_object.c:202:osc_object_ast_clear()) ### update lock size 0 blocks 0 [cma]time: 0 0 0 by oinfo size 0 blocks 0 [cma]time 0 0 0 ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 0000000005bd4a7e/0xef30c340bfb3d038 lrc: 1/0,0 mode: PR/PR res: [0x2c0000401:0x5e5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0xdb7edb52b82f68c4 expref: -99 pid: 78193 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571772.321213:0:78193:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571772.321388:0:78193:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571772.321402:0:78193:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 30 res 8589934599/1 (0 0) 00010000:00010000:0.0:1788571772.321472:0:78193:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d077 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x10000000000000 pid: 78193 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.321484:0:78193:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d077 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x0 pid: 78193 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.321498:0:78193:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d077 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x0 pid: 78193 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571772.321538:0:78193:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f2d80 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:78193:1875449613563136:192.168.202.113@tcp:101:rm.0 00000100:00100000:0.0:1788571772.321713:0:78193:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada465f4000 going to sleep for 11 seconds 00000100:00100000:0.0:1788571772.336797:0:78193:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f2d80 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:78193:1875449613563136:192.168.202.113@tcp:101:rm.0 00010000:00010000:0.0:1788571772.336849:0:78193:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d077 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6918 expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.336913:0:78193:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d077 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6918 expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.336928:0:78193:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d077 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6918 expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.336942:0:78193:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d077 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6918 expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.336960:0:78193:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d077 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6918 expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.336970:0:78193:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d077 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6918 expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.336978:0:78193:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d077 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6918 expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788571772.336994:0:78193:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada516f2d80 x1875449613563136/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788571788 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788571772.337012:0:78193:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d077 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6918 expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.337028:0:78193:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1788571772.337038:0:78193:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1788571772.337079:0:78193:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:0.0:1788571772.337086:0:78193:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d077 00000080:00010000:0.0:1788571772.337093:0:78193:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571772.337097:0:78193:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d077 from it ffffab0ec1cefb58 00010000:00010000:0.0:1788571772.337100:0:78193:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d077 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6918 expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.337111:0:78193:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d077 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6918 expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571772.337143:0:78193:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:0.0:1788571772.337161:0:78193:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33a.replay-single--7, flags=8 00010000:00010000:0.0:1788571772.337183:0:78193:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f200/0xef30c340bfb3cfba lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f6720 expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00010000:0.0:1788571772.337191:0:78193:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cfba from it ffffab0ec1cefaf8 00010000:00010000:0.0:1788571772.337194:0:78193:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f200/0xef30c340bfb3cfba lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f6720 expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571772.337201:0:78193:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f200/0xef30c340bfb3cfba lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f6720 expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200008:0.0:1788571772.337489:0:78193:0:(file.c:6296:ll_getattr_dentry()) START file f33a.replay-single--7:[0x200015bc1:0x8:0x0](ffff9ada582542c8), request_mask 2047, flags 0, foreign 0 00010000:00010000:2.0:1788571772.339409:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9600/0xef30c340bfb3d07e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.339426:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9600/0xef30c340bfb3d07e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.339460:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8c00/0xef30c340bfb3d085 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.339467:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8c00/0xef30c340bfb3d085 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.339485:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9200/0xef30c340bfb3d08c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.339491:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9200/0xef30c340bfb3d08c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.339511:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9000/0xef30c340bfb3d093 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.339517:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9000/0xef30c340bfb3d093 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.339533:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3d09a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.339539:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3d09a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.339556:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9400/0xef30c340bfb3d0a1 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.339563:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9400/0xef30c340bfb3d0a1 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.339581:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3d0a8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.339588:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3d0a8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.339608:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8e00/0xef30c340bfb3d0af lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.339626:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8e00/0xef30c340bfb3d0af lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 9 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.339647:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8600/0xef30c340bfb3d0b6 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 10 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.339655:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8600/0xef30c340bfb3d0b6 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 10 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571772.339742:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1788571772.339765:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613563520:192.168.202.113@tcp:63:ll_sa.0 00010000:00010000:2.0:1788571772.376171:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3d0c4 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.376181:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3d0c4 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.376211:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3d0cb lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.376218:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3d0cb lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.376369:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9c00/0xef30c340bfb3d0d2 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.376377:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9c00/0xef30c340bfb3d0d2 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.376399:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9800/0xef30c340bfb3d0d9 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.376406:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9800/0xef30c340bfb3d0d9 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.376424:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8400/0xef30c340bfb3d0e0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.376431:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8400/0xef30c340bfb3d0e0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.376449:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9e00/0xef30c340bfb3d0e7 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.376455:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9e00/0xef30c340bfb3d0e7 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.376474:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8600/0xef30c340bfb3d0ee lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.376480:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8600/0xef30c340bfb3d0ee lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.376496:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8e00/0xef30c340bfb3d0f5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.376502:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8e00/0xef30c340bfb3d0f5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 9 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.376519:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3d0fc lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 10 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.376525:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3d0fc lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 10 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.376542:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9400/0xef30c340bfb3d103 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 11 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.376548:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9400/0xef30c340bfb3d103 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.376569:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3d10a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 12 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.376576:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3d10a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.376592:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9000/0xef30c340bfb3d111 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.376599:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9000/0xef30c340bfb3d111 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 13 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.376631:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9200/0xef30c340bfb3d118 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 14 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.376638:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9200/0xef30c340bfb3d118 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.376660:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8c00/0xef30c340bfb3d11f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 15 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.376667:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8c00/0xef30c340bfb3d11f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.376684:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9600/0xef30c340bfb3d126 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 16 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.376691:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9600/0xef30c340bfb3d126 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.376709:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0200/0xef30c340bfb3d12d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 17 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.376717:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0200/0xef30c340bfb3d12d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.376734:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d134 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 18 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.376740:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d134 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571772.376814:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1788571772.376834:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a34700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613564032:192.168.202.113@tcp:63:ll_sa.0 00000100:00100000:0.0:1788571772.377304:0:78193:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada5195b180 going to sleep for 11 seconds 00010000:00010000:2.0:1788571772.384774:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3d0c4 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.384814:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3d0c4 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.384822:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3d0c4 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.384829:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3d0c4 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.384839:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3d0c4 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.384845:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada603e8200 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.384850:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3d0c4 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385209:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3d0c4 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385217:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3d0c4 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385224:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.385233:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3d0c4 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385241:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8200/0xef30c340bfb3d0c4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385269:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3d0cb lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385276:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3d0cb lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385283:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3d0cb lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385289:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3d0cb lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385295:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3d0cb lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385301:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada603e9a00 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.385304:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3d0cb lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385314:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3d0cb lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385321:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3d0cb lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385327:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.385330:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3d0cb lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385335:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9a00/0xef30c340bfb3d0cb lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385347:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9c00/0xef30c340bfb3d0d2 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385353:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9c00/0xef30c340bfb3d0d2 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385359:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9c00/0xef30c340bfb3d0d2 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385363:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9c00/0xef30c340bfb3d0d2 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385369:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9c00/0xef30c340bfb3d0d2 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385373:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada603e9c00 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.385375:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9c00/0xef30c340bfb3d0d2 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385384:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9c00/0xef30c340bfb3d0d2 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385389:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9c00/0xef30c340bfb3d0d2 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385393:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.385395:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9c00/0xef30c340bfb3d0d2 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385400:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9c00/0xef30c340bfb3d0d2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385411:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9800/0xef30c340bfb3d0d9 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385417:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9800/0xef30c340bfb3d0d9 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385423:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9800/0xef30c340bfb3d0d9 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385429:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9800/0xef30c340bfb3d0d9 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385434:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9800/0xef30c340bfb3d0d9 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385438:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada603e9800 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.385441:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9800/0xef30c340bfb3d0d9 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385449:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9800/0xef30c340bfb3d0d9 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385455:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9800/0xef30c340bfb3d0d9 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385459:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.385461:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9800/0xef30c340bfb3d0d9 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385467:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9800/0xef30c340bfb3d0d9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385477:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8400/0xef30c340bfb3d0e0 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385483:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8400/0xef30c340bfb3d0e0 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385489:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8400/0xef30c340bfb3d0e0 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385494:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8400/0xef30c340bfb3d0e0 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385499:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8400/0xef30c340bfb3d0e0 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385504:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada603e8400 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.385506:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8400/0xef30c340bfb3d0e0 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 14 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385514:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8400/0xef30c340bfb3d0e0 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 14 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385520:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8400/0xef30c340bfb3d0e0 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 14 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385525:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.385528:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8400/0xef30c340bfb3d0e0 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 14 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385535:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8400/0xef30c340bfb3d0e0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 14 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385547:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9e00/0xef30c340bfb3d0e7 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 13 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385553:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9e00/0xef30c340bfb3d0e7 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 13 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385558:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9e00/0xef30c340bfb3d0e7 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 13 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385564:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9e00/0xef30c340bfb3d0e7 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 13 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385569:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9e00/0xef30c340bfb3d0e7 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 13 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385574:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada603e9e00 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.385577:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9e00/0xef30c340bfb3d0e7 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 13 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385586:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9e00/0xef30c340bfb3d0e7 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 13 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385592:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9e00/0xef30c340bfb3d0e7 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 13 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385598:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.385602:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9e00/0xef30c340bfb3d0e7 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 13 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385609:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9e00/0xef30c340bfb3d0e7 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 13 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385631:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8600/0xef30c340bfb3d0ee lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385638:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8600/0xef30c340bfb3d0ee lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385644:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8600/0xef30c340bfb3d0ee lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385650:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8600/0xef30c340bfb3d0ee lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385656:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8600/0xef30c340bfb3d0ee lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385662:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada603e8600 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.385664:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8600/0xef30c340bfb3d0ee lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 12 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385673:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8600/0xef30c340bfb3d0ee lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 12 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385680:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8600/0xef30c340bfb3d0ee lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 12 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385685:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.385688:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8600/0xef30c340bfb3d0ee lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 12 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385695:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8600/0xef30c340bfb3d0ee lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 12 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385707:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8e00/0xef30c340bfb3d0f5 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385712:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8e00/0xef30c340bfb3d0f5 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385717:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8e00/0xef30c340bfb3d0f5 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385723:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8e00/0xef30c340bfb3d0f5 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385729:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8e00/0xef30c340bfb3d0f5 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385734:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada603e8e00 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.385738:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8e00/0xef30c340bfb3d0f5 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385747:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8e00/0xef30c340bfb3d0f5 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385752:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8e00/0xef30c340bfb3d0f5 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385758:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.385761:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8e00/0xef30c340bfb3d0f5 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385767:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8e00/0xef30c340bfb3d0f5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385780:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3d0fc lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 10 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385786:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3d0fc lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 10 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385791:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3d0fc lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 10 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385797:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3d0fc lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 10 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385820:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3d0fc lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 10 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385826:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada603e8a00 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.385828:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3d0fc lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 10 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385838:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3d0fc lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 10 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385844:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3d0fc lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 10 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.385849:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.386422:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3d0fc lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 10 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386436:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8a00/0xef30c340bfb3d0fc lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 10 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386455:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9400/0xef30c340bfb3d103 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 9 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386462:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9400/0xef30c340bfb3d103 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 9 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386469:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9400/0xef30c340bfb3d103 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 9 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386476:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9400/0xef30c340bfb3d103 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 9 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386482:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9400/0xef30c340bfb3d103 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 9 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386488:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada603e9400 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.386492:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9400/0xef30c340bfb3d103 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 9 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386501:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9400/0xef30c340bfb3d103 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 9 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386507:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9400/0xef30c340bfb3d103 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 9 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386514:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.386518:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9400/0xef30c340bfb3d103 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 9 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386524:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9400/0xef30c340bfb3d103 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 9 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386537:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3d10a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386543:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3d10a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386548:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3d10a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386555:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3d10a lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386562:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3d10a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386567:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada603e8800 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.386570:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3d10a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 8 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386580:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3d10a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 8 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386585:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3d10a lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 8 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386591:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.386594:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3d10a lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 8 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386599:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8800/0xef30c340bfb3d10a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 8 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386612:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9000/0xef30c340bfb3d111 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386629:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9000/0xef30c340bfb3d111 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386635:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9000/0xef30c340bfb3d111 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386641:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9000/0xef30c340bfb3d111 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386648:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9000/0xef30c340bfb3d111 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386653:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada603e9000 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.386656:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9000/0xef30c340bfb3d111 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 7 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386665:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9000/0xef30c340bfb3d111 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 7 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386671:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9000/0xef30c340bfb3d111 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 7 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386676:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.386679:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9000/0xef30c340bfb3d111 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 7 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386685:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9000/0xef30c340bfb3d111 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 7 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386698:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9200/0xef30c340bfb3d118 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386704:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9200/0xef30c340bfb3d118 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386710:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9200/0xef30c340bfb3d118 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386717:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9200/0xef30c340bfb3d118 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386723:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9200/0xef30c340bfb3d118 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386728:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada603e9200 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.386731:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9200/0xef30c340bfb3d118 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 6 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386739:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9200/0xef30c340bfb3d118 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 6 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386744:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9200/0xef30c340bfb3d118 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 6 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386749:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.386752:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9200/0xef30c340bfb3d118 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 6 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386758:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9200/0xef30c340bfb3d118 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 6 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386770:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8c00/0xef30c340bfb3d11f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386776:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8c00/0xef30c340bfb3d11f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386781:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8c00/0xef30c340bfb3d11f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386785:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8c00/0xef30c340bfb3d11f lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386790:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8c00/0xef30c340bfb3d11f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386795:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada603e8c00 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.386809:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8c00/0xef30c340bfb3d11f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 5 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386820:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8c00/0xef30c340bfb3d11f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 5 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386826:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8c00/0xef30c340bfb3d11f lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 5 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386831:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.386835:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8c00/0xef30c340bfb3d11f lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 5 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.386840:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8c00/0xef30c340bfb3d11f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 5 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.387731:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9600/0xef30c340bfb3d126 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.387748:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9600/0xef30c340bfb3d126 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.387755:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9600/0xef30c340bfb3d126 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.387761:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9600/0xef30c340bfb3d126 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.387767:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9600/0xef30c340bfb3d126 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.387772:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada603e9600 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.387776:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9600/0xef30c340bfb3d126 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.387785:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9600/0xef30c340bfb3d126 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.387792:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9600/0xef30c340bfb3d126 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.387984:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.387995:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9600/0xef30c340bfb3d126 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.388004:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9600/0xef30c340bfb3d126 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.388027:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0200/0xef30c340bfb3d12d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.388034:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0200/0xef30c340bfb3d12d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.388040:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0200/0xef30c340bfb3d12d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.388046:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0200/0xef30c340bfb3d12d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.388051:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0200/0xef30c340bfb3d12d lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.388056:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada516e0200 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.388059:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0200/0xef30c340bfb3d12d lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 3 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.388068:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0200/0xef30c340bfb3d12d lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 3 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.388073:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0200/0xef30c340bfb3d12d lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 3 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.388078:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.388081:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0200/0xef30c340bfb3d12d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.388088:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0200/0xef30c340bfb3d12d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.388102:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d134 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.388109:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d134 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.388115:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d134 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.388121:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d134 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.388128:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d134 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.388133:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada516e1000 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.388136:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d134 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 2 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.388145:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d134 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 2 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.388151:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d134 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 2 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.388157:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.388161:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d134 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.388167:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d134 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571772.388207:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a34700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613564032:192.168.202.113@tcp:63:ll_sa.0 00000100:00100000:0.0:1788571772.388403:0:78193:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f6a00 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:78193:1875449613563904:192.168.202.113@tcp:36:rm.0 00000080:00200000:0.0:1788571772.388474:0:78193:0:(vvp_io.c:1826:vvp_io_init()) [0x200015bc1:0x9:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1788571772.388491:0:78193:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1788571772.388533:0:78193:0:(vvp_io.c:304:vvp_io_fini()) [0x200015bc1:0x9:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1788571772.388543:0:78193:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200015bc1:0x9:0x0](ffff9ada58252a08) 00000008:00010000:0.0:1788571772.388586:0:78193:0:(osc_object.c:202:osc_object_ast_clear()) ### update lock size 0 blocks 0 [cma]time: 0 0 0 by oinfo size 0 blocks 0 [cma]time 0 0 0 ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 000000009657ad4d/0xef30c340bfb3d046 lrc: 1/0,0 mode: PR/PR res: [0x2c0000401:0x5e6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0xdb7edb52b82f68d2 expref: -99 pid: 78195 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788571772.388705:0:78193:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571772.388738:0:78193:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571772.389107:0:78193:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617800/0xef30c340bfb3d13b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78193 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.389117:0:78193:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617800/0xef30c340bfb3d13b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78193 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.389130:0:78193:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617800/0xef30c340bfb3d13b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78193 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571772.389161:0:78193:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f2d80 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:78193:1875449613564160:192.168.202.113@tcp:101:rm.0 00000100:00100000:0.0:1788571772.389270:0:78193:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada465f4000 going to sleep for 11 seconds 00000100:00100000:0.0:1788571772.392207:0:78193:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f2d80 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:78193:1875449613564160:192.168.202.113@tcp:101:rm.0 00010000:00010000:0.0:1788571772.392253:0:78193:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617800/0xef30c340bfb3d13b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6996 expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.392268:0:78193:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617800/0xef30c340bfb3d13b lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6996 expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.392283:0:78193:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617800/0xef30c340bfb3d13b lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6996 expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.392297:0:78193:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617800/0xef30c340bfb3d13b lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6996 expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.392313:0:78193:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617800/0xef30c340bfb3d13b lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6996 expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.392322:0:78193:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617800/0xef30c340bfb3d13b lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6996 expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.392331:0:78193:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617800/0xef30c340bfb3d13b lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6996 expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788571772.392343:0:78193:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada516f2d80 x1875449613564160/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788571788 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788571772.392358:0:78193:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617800/0xef30c340bfb3d13b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6996 expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.392374:0:78193:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1788571772.392382:0:78193:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1788571772.392416:0:78193:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:0.0:1788571772.392423:0:78193:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d13b 00000080:00010000:0.0:1788571772.392429:0:78193:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571772.392432:0:78193:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d13b from it ffffab0ec1cefb58 00010000:00010000:0.0:1788571772.392436:0:78193:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617800/0xef30c340bfb3d13b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6996 expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571772.392446:0:78193:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617800/0xef30c340bfb3d13b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6996 expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571772.392480:0:78193:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:0.0:1788571772.392493:0:78193:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33a.replay-single--9, flags=8 00010000:00010000:0.0:1788571772.392512:0:78193:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616400/0xef30c340bfb3cfd6 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f6782 expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00010000:0.0:1788571772.392520:0:78193:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cfd6 from it ffffab0ec1cefaf8 00010000:00010000:0.0:1788571772.392523:0:78193:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616400/0xef30c340bfb3cfd6 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f6782 expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571772.392531:0:78193:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616400/0xef30c340bfb3cfd6 lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f6782 expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200008:0.0:1788571772.392565:0:78193:0:(file.c:6296:ll_getattr_dentry()) START file f33a.replay-single--9:[0x200015bc1:0xa:0x0](ffff9ada5825f448), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1788571772.392573:0:78193:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200015bc1:0xa:0x0](ffff9ada5825f448),name=f33a.replay-single--9 00000002:00010000:0.0:1788571772.392600:0:78193:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200015bc1:0xa:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571772.392610:0:78193:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616400/0xef30c340bfb3cfd6 lrc: 2/1,0 mode: PR/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f6782 expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1788571772.392635:0:78193:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616400/0xef30c340bfb3cfd6 lrc: 2/1,0 mode: PR/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f6782 expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1788571772.392643:0:78193:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788571772.392648:0:78193:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0xa:0x0](ffff9ada5825f448) 00000080:00010000:0.0:1788571772.392651:0:78193:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0xa:0x0](ffff9ada5825f448) for lock 0xef30c340bfb3cfd6 00000080:00010000:0.0:1788571772.392655:0:78193:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3cfd6 from it ffffab0ec1cefcb8 00010000:00010000:0.0:1788571772.392658:0:78193:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616400/0xef30c340bfb3cfd6 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f6782 expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1788571772.392668:0:78193:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616400/0xef30c340bfb3cfd6 lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f6782 expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 4 lru_type: 0 00000080:00200000:0.0:1788571772.392700:0:78193:0:(vvp_io.c:1826:vvp_io_init()) [0x200015bc1:0xa:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:0.0:1788571772.392715:0:78193:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x200015bc1:0xa:0x0] 00000008:00010000:0.0:1788571772.392749:0:78193:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a 00000008:00010000:0.0:1788571772.392754:0:78193:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff9ada489953b0, osc lock ffff9ada7d07abe0, flags 0x400001002 00010000:00010000:0.0:1788571772.392779:0:78193:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 000000002bdeef89/0xef30c340bfb3d04d lrc: 2/1,0 mode: PR/PR res: [0x280000401:0x5e6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f68d9 expref: -99 pid: 78195 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571772.392793:0:78193:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 18446744073709551615) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 000000002bdeef89/0xef30c340bfb3d04d lrc: 2/1,0 mode: PR/PR res: [0x280000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f68d9 expref: -99 pid: 78195 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000008:00010000:0.0:1788571772.392809:0:78193:0:(osc_lock.c:187:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 000000002bdeef89/0xef30c340bfb3d04d lrc: 4/1,0 mode: PR/PR res: [0x280000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f68d9 expref: -99 pid: 78195 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00200000:0.0:1788571772.392821:0:78193:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00010000:00010000:0.0:1788571772.392825:0:78193:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 000000002bdeef89/0xef30c340bfb3d04d lrc: 5/1,0 mode: PR/PR res: [0x280000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0xdb7edb52b82f68d9 expref: -99 pid: 78195 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571772.392835:0:78193:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 000000002bdeef89/0xef30c340bfb3d04d lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0xdb7edb52b82f68d9 expref: -99 pid: 78195 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000020:00200000:0.0:1788571772.392846:0:78193:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1788571772.392898:0:78193:0:(file.c:1775:ll_merge_attr_nolock()) [0x200015bc1:0xa:0x0] updating i_size 0 i_blocks 0 00000020:00010000:0.0:1788571772.392905:0:78193:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9ada60bc06f0 (ffff9ada60b0d2d8/0) at cl_lock_release():245 00000020:00010000:0.0:1788571772.392909:0:78193:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9ada60bc06f0 (ffff9ada60b0d2d8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1788571772.392913:0:78193:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9ada489953b0 (ffff9ada60b68ea8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1788571772.392919:0:78193:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9ada60bc06f0 (ffff9ada60b0d2d8/0) at cl_lock_fini():72 00000020:00010000:0.0:1788571772.392922:0:78193:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9ada489953b0 (ffff9ada60b0d2d8/1) at cl_lock_fini():72 00000080:00200000:0.0:1788571772.392937:0:78193:0:(vvp_io.c:304:vvp_io_fini()) [0x200015bc1:0xa:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1788571772.392979:0:78193:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571772.392988:0:78193:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571772.392995:0:78193:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617800/0xef30c340bfb3d13b lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6996 expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571772.393005:0:78193:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617800/0xef30c340bfb3d13b lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6996 expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571772.393011:0:78193:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788571772.393015:0:78193:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571772.393018:0:78193:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d13b 00000080:00010000:0.0:1788571772.393022:0:78193:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d13b from it ffffab0ec1cefc10 00010000:00010000:0.0:1788571772.393025:0:78193:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617800/0xef30c340bfb3d13b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6996 expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571772.393032:0:78193:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617800/0xef30c340bfb3d13b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6996 expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788571772.393041:0:78193:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:0.0:1788571772.393050:0:78193:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33a.replay-single--9, flags=0 00000080:00200000:0.0:1788571772.393055:0:78193:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571772.393061:0:78193:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571772.393068:0:78193:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617800/0xef30c340bfb3d13b lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6996 expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571772.393076:0:78193:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617800/0xef30c340bfb3d13b lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6996 expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788571772.393083:0:78193:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788571772.393087:0:78193:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571772.393090:0:78193:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d13b 00000080:00010000:0.0:1788571772.393093:0:78193:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d13b from it ffffab0ec1cefd68 00010000:00010000:0.0:1788571772.393096:0:78193:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617800/0xef30c340bfb3d13b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6996 expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571772.393103:0:78193:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617800/0xef30c340bfb3d13b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6996 expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1788571772.393112:0:78193:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:0.0:1788571772.393120:0:78193:0:(namei.c:2351:ll_unlink()) VFS Op:name=f33a.replay-single--9, dir=[0x200000007:0x1:0x0](ffff9ada58125b88) 00010000:00010000:0.0:1788571772.393149:0:78193:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617800/0xef30c340bfb3d13b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b82f6996 expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1788571772.393176:0:78193:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616400/0xef30c340bfb3cfd6 lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0xdb7edb52b82f6782 expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 4 lru_type: 0 00000080:00200000:0.0:1788571772.393189:0:78193:0:(vvp_object.c:117:vvp_conf_set()) [0x200015bc1:0xa:0x0]: losing layout lock 00010000:00010000:0.0:1788571772.393238:0:78193:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616400/0xef30c340bfb3cfd6 lrc: 1/0,0 mode: --/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0xdb7edb52b82f6782 expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 4 lru_type: 0 00010000:00010000:0.0:1788571772.393245:0:78193:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617800/0xef30c340bfb3d13b lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f6996 expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1788571772.393252:0:78193:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:0.0:1788571772.393256:0:78193:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616400/0xef30c340bfb3cfd6 lrc: 0/0,0 mode: --/PR res: [0x200015bc1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0xdb7edb52b82f6782 expref: -99 pid: 78192 timeout: 0 lvb_type: 3 lru_score: 4 lru_type: 0 00010000:00010000:0.0:1788571772.393278:0:78193:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617800/0xef30c340bfb3d13b lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f6996 expref: -99 pid: 78193 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:0.0:1788571772.393303:0:78193:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1788571772.393314:0:78193:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada516f2d80 x1875449613564416/t0(0) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788571772.393341:0:78193:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f2d80 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:78193:1875449613564416:192.168.202.113@tcp:36:rm.0 00000100:00100000:0.0:1788571772.393437:0:78193:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada465f4000 going to sleep for 11 seconds 00010000:00010000:2.0:1788571772.393727:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1e00/0xef30c340bfb3d142 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.393737:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1e00/0xef30c340bfb3d142 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.393760:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0e00/0xef30c340bfb3d149 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.393766:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0e00/0xef30c340bfb3d149 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.393779:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d150 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.393784:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d150 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.393800:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1c00/0xef30c340bfb3d157 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.393806:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1c00/0xef30c340bfb3d157 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.393818:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d15e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.393824:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d15e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.393836:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0600/0xef30c340bfb3d165 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.393842:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0600/0xef30c340bfb3d165 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.393882:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0400/0xef30c340bfb3d16c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.393888:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0400/0xef30c340bfb3d16c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.393902:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0800/0xef30c340bfb3d173 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.393907:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0800/0xef30c340bfb3d173 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 9 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.393922:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0a00/0xef30c340bfb3d17a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 10 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.393929:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0a00/0xef30c340bfb3d17a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 10 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.393944:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1a00/0xef30c340bfb3d181 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 11 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.393950:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1a00/0xef30c340bfb3d181 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.393967:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1800/0xef30c340bfb3d188 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 12 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.393973:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1800/0xef30c340bfb3d188 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.393987:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1400/0xef30c340bfb3d18f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.393992:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1400/0xef30c340bfb3d18f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 13 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394016:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1200/0xef30c340bfb3d196 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 14 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394023:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1200/0xef30c340bfb3d196 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394040:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0c00/0xef30c340bfb3d19d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 15 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394046:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0c00/0xef30c340bfb3d19d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394062:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16200/0xef30c340bfb3d1a4 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 16 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394069:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16200/0xef30c340bfb3d1a4 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394084:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16800/0xef30c340bfb3d1ab lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 17 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394090:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16800/0xef30c340bfb3d1ab lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394104:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16400/0xef30c340bfb3d1b2 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 18 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394109:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16400/0xef30c340bfb3d1b2 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394126:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17600/0xef30c340bfb3d1b9 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 19 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394133:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17600/0xef30c340bfb3d1b9 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 19 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394149:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17c00/0xef30c340bfb3d1c0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 20 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394156:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17c00/0xef30c340bfb3d1c0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 20 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394174:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16a00/0xef30c340bfb3d1c7 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 21 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394181:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16a00/0xef30c340bfb3d1c7 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 21 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394200:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17000/0xef30c340bfb3d1ce lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 22 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394206:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17000/0xef30c340bfb3d1ce lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 22 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394226:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16600/0xef30c340bfb3d1d5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 23 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394292:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16600/0xef30c340bfb3d1d5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 23 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394318:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17200/0xef30c340bfb3d1dc lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 24 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394325:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17200/0xef30c340bfb3d1dc lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 24 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394368:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16e00/0xef30c340bfb3d1e3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 25 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394374:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16e00/0xef30c340bfb3d1e3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 25 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394390:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16000/0xef30c340bfb3d1ea lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 26 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394397:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16000/0xef30c340bfb3d1ea lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 26 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394413:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17800/0xef30c340bfb3d1f1 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 27 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394419:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17800/0xef30c340bfb3d1f1 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 27 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394433:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17a00/0xef30c340bfb3d1f8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 28 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394439:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17a00/0xef30c340bfb3d1f8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 28 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394458:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16c00/0xef30c340bfb3d1ff lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 29 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394463:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16c00/0xef30c340bfb3d1ff lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 29 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394478:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17400/0xef30c340bfb3d206 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 30 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394485:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17400/0xef30c340bfb3d206 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 30 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394498:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17e00/0xef30c340bfb3d20d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 31 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394504:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17e00/0xef30c340bfb3d20d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 31 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394522:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c000/0xef30c340bfb3d214 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 32 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394528:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c000/0xef30c340bfb3d214 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 32 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394545:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d000/0xef30c340bfb3d21b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 33 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394552:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d000/0xef30c340bfb3d21b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 33 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394571:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d400/0xef30c340bfb3d222 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 34 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.394577:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d400/0xef30c340bfb3d222 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 34 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571772.394674:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a34700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613564544:192.168.202.113@tcp:63:ll_sa.0 00010000:00010000:2.0:1788571772.401669:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1e00/0xef30c340bfb3d142 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 34 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.401693:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1e00/0xef30c340bfb3d142 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 34 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.401701:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1e00/0xef30c340bfb3d142 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 34 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.401709:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1e00/0xef30c340bfb3d142 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 34 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.401718:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1e00/0xef30c340bfb3d142 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 34 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.401729:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada516e1e00 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.401736:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1e00/0xef30c340bfb3d142 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 34 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.401743:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1e00/0xef30c340bfb3d142 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 34 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.401749:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1e00/0xef30c340bfb3d142 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 34 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.401757:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.401765:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1e00/0xef30c340bfb3d142 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 34 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.401772:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1e00/0xef30c340bfb3d142 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 34 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.401808:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0e00/0xef30c340bfb3d149 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 33 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.401814:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0e00/0xef30c340bfb3d149 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 33 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.401819:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0e00/0xef30c340bfb3d149 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 33 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.401825:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0e00/0xef30c340bfb3d149 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 33 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.401830:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0e00/0xef30c340bfb3d149 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 33 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.401837:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada516e0e00 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.401840:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0e00/0xef30c340bfb3d149 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 33 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.401846:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0e00/0xef30c340bfb3d149 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 33 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.401901:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0e00/0xef30c340bfb3d149 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 33 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.401906:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.401910:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0e00/0xef30c340bfb3d149 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 33 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.401916:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0e00/0xef30c340bfb3d149 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 33 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.401931:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d150 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 32 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.401938:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d150 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 32 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.401943:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d150 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 32 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.401948:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d150 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 32 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.401953:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d150 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 32 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.401959:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada516e1600 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.401961:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d150 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 32 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.401967:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d150 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 32 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.401972:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d150 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 32 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.401977:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.401980:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d150 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 32 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.401985:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d150 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 32 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.401996:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1c00/0xef30c340bfb3d157 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 31 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402002:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1c00/0xef30c340bfb3d157 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 31 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402007:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1c00/0xef30c340bfb3d157 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 31 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402012:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1c00/0xef30c340bfb3d157 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 31 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402019:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1c00/0xef30c340bfb3d157 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 31 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402025:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada516e1c00 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.402027:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1c00/0xef30c340bfb3d157 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 31 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402033:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1c00/0xef30c340bfb3d157 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 31 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402039:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1c00/0xef30c340bfb3d157 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 31 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402045:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.402047:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1c00/0xef30c340bfb3d157 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 31 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402053:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1c00/0xef30c340bfb3d157 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 31 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402065:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d15e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 30 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402071:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d15e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 30 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402077:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d15e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 30 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402083:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d15e lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 30 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402088:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d15e lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 30 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402094:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada516e0000 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.402096:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d15e lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 30 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402101:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d15e lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 30 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402107:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d15e lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 30 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402113:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.402116:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d15e lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 30 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402122:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d15e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 30 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402134:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0600/0xef30c340bfb3d165 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 29 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402139:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0600/0xef30c340bfb3d165 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 29 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402145:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0600/0xef30c340bfb3d165 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 29 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402151:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0600/0xef30c340bfb3d165 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 29 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402157:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0600/0xef30c340bfb3d165 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 29 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402164:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada516e0600 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.402167:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0600/0xef30c340bfb3d165 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 29 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402173:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0600/0xef30c340bfb3d165 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 29 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402179:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0600/0xef30c340bfb3d165 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 29 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402184:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.402187:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0600/0xef30c340bfb3d165 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 29 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402194:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0600/0xef30c340bfb3d165 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 29 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402205:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0400/0xef30c340bfb3d16c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 28 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402217:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0400/0xef30c340bfb3d16c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 28 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402223:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0400/0xef30c340bfb3d16c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 28 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402230:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0400/0xef30c340bfb3d16c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 28 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402237:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0400/0xef30c340bfb3d16c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 28 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402242:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada516e0400 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.402245:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0400/0xef30c340bfb3d16c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 28 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402251:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0400/0xef30c340bfb3d16c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 28 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402257:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0400/0xef30c340bfb3d16c lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 28 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402262:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.402265:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0400/0xef30c340bfb3d16c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 28 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402271:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0400/0xef30c340bfb3d16c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 28 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402285:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0800/0xef30c340bfb3d173 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 27 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402291:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0800/0xef30c340bfb3d173 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 27 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402296:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0800/0xef30c340bfb3d173 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 27 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402302:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0800/0xef30c340bfb3d173 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 27 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402307:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0800/0xef30c340bfb3d173 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 27 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402314:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada516e0800 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.402317:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0800/0xef30c340bfb3d173 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 27 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402323:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0800/0xef30c340bfb3d173 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 27 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402329:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0800/0xef30c340bfb3d173 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 27 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402335:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.402338:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0800/0xef30c340bfb3d173 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 27 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402344:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0800/0xef30c340bfb3d173 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 27 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402356:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0a00/0xef30c340bfb3d17a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 26 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402362:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0a00/0xef30c340bfb3d17a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 26 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402368:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0a00/0xef30c340bfb3d17a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 26 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402374:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0a00/0xef30c340bfb3d17a lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 26 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402380:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0a00/0xef30c340bfb3d17a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 26 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402385:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada516e0a00 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.402388:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0a00/0xef30c340bfb3d17a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 26 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402394:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0a00/0xef30c340bfb3d17a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 26 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402400:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0a00/0xef30c340bfb3d17a lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 26 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402405:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.402408:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0a00/0xef30c340bfb3d17a lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 26 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402413:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0a00/0xef30c340bfb3d17a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 26 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402425:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1a00/0xef30c340bfb3d181 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 25 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402431:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1a00/0xef30c340bfb3d181 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 25 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402436:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1a00/0xef30c340bfb3d181 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 25 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402443:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1a00/0xef30c340bfb3d181 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 25 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402448:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1a00/0xef30c340bfb3d181 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 25 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402453:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada516e1a00 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.402455:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1a00/0xef30c340bfb3d181 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 25 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402460:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1a00/0xef30c340bfb3d181 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 25 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402464:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1a00/0xef30c340bfb3d181 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 25 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402468:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.402471:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1a00/0xef30c340bfb3d181 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 25 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402476:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1a00/0xef30c340bfb3d181 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 25 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402486:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1800/0xef30c340bfb3d188 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 24 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402491:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1800/0xef30c340bfb3d188 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 24 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402497:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1800/0xef30c340bfb3d188 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 24 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402502:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1800/0xef30c340bfb3d188 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 24 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402506:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1800/0xef30c340bfb3d188 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 24 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402511:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada516e1800 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.402515:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1800/0xef30c340bfb3d188 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 24 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402521:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1800/0xef30c340bfb3d188 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 24 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402527:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1800/0xef30c340bfb3d188 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 24 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402531:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.402534:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1800/0xef30c340bfb3d188 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 24 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402539:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1800/0xef30c340bfb3d188 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 24 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402550:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1400/0xef30c340bfb3d18f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 23 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402557:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1400/0xef30c340bfb3d18f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 23 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402563:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1400/0xef30c340bfb3d18f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 23 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402569:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1400/0xef30c340bfb3d18f lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 23 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402575:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1400/0xef30c340bfb3d18f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 23 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402580:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada516e1400 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.402584:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1400/0xef30c340bfb3d18f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 23 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402589:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1400/0xef30c340bfb3d18f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 23 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402595:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1400/0xef30c340bfb3d18f lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 23 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402600:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.402604:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1400/0xef30c340bfb3d18f lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 23 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402611:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1400/0xef30c340bfb3d18f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 23 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402636:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1200/0xef30c340bfb3d196 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 22 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402643:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1200/0xef30c340bfb3d196 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 22 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402648:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1200/0xef30c340bfb3d196 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 22 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402654:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1200/0xef30c340bfb3d196 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 22 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402660:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1200/0xef30c340bfb3d196 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 22 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402666:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada516e1200 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.402669:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1200/0xef30c340bfb3d196 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 22 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402675:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1200/0xef30c340bfb3d196 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 22 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402681:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1200/0xef30c340bfb3d196 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 22 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402686:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.402689:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1200/0xef30c340bfb3d196 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 22 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402696:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1200/0xef30c340bfb3d196 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 22 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402708:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0c00/0xef30c340bfb3d19d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 21 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402715:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0c00/0xef30c340bfb3d19d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 21 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402720:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0c00/0xef30c340bfb3d19d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 21 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402725:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0c00/0xef30c340bfb3d19d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 21 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402731:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0c00/0xef30c340bfb3d19d lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 21 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402736:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada516e0c00 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.402739:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0c00/0xef30c340bfb3d19d lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 21 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402744:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0c00/0xef30c340bfb3d19d lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 21 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402749:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0c00/0xef30c340bfb3d19d lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 21 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402754:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.402757:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0c00/0xef30c340bfb3d19d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 21 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402762:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0c00/0xef30c340bfb3d19d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 21 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402773:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16200/0xef30c340bfb3d1a4 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 20 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402777:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16200/0xef30c340bfb3d1a4 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 20 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402782:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16200/0xef30c340bfb3d1a4 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 20 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402788:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16200/0xef30c340bfb3d1a4 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 20 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402792:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16200/0xef30c340bfb3d1a4 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 20 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402799:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada60a16200 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.402802:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16200/0xef30c340bfb3d1a4 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 20 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402806:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16200/0xef30c340bfb3d1a4 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 20 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402811:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16200/0xef30c340bfb3d1a4 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 20 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402816:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.402819:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16200/0xef30c340bfb3d1a4 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 20 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402826:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16200/0xef30c340bfb3d1a4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 20 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402838:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16800/0xef30c340bfb3d1ab lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 19 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402844:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16800/0xef30c340bfb3d1ab lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 19 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402849:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16800/0xef30c340bfb3d1ab lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 19 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402895:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16800/0xef30c340bfb3d1ab lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 19 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402901:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16800/0xef30c340bfb3d1ab lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 19 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402906:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada60a16800 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.402909:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16800/0xef30c340bfb3d1ab lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 19 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402914:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16800/0xef30c340bfb3d1ab lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 19 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402920:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16800/0xef30c340bfb3d1ab lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 19 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402925:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.402928:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16800/0xef30c340bfb3d1ab lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 19 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402933:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16800/0xef30c340bfb3d1ab lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 19 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402945:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16400/0xef30c340bfb3d1b2 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402951:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16400/0xef30c340bfb3d1b2 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402957:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16400/0xef30c340bfb3d1b2 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402963:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16400/0xef30c340bfb3d1b2 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402968:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16400/0xef30c340bfb3d1b2 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402973:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada60a16400 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.402975:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16400/0xef30c340bfb3d1b2 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402981:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16400/0xef30c340bfb3d1b2 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402986:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16400/0xef30c340bfb3d1b2 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402990:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.402993:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16400/0xef30c340bfb3d1b2 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.402998:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16400/0xef30c340bfb3d1b2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403010:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17600/0xef30c340bfb3d1b9 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403016:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17600/0xef30c340bfb3d1b9 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403021:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17600/0xef30c340bfb3d1b9 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403028:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17600/0xef30c340bfb3d1b9 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403033:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17600/0xef30c340bfb3d1b9 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403038:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada60a17600 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.403041:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17600/0xef30c340bfb3d1b9 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403046:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17600/0xef30c340bfb3d1b9 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403052:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17600/0xef30c340bfb3d1b9 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403058:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.403061:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17600/0xef30c340bfb3d1b9 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403067:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17600/0xef30c340bfb3d1b9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403080:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17c00/0xef30c340bfb3d1c0 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403087:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17c00/0xef30c340bfb3d1c0 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403092:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17c00/0xef30c340bfb3d1c0 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403098:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17c00/0xef30c340bfb3d1c0 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403104:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17c00/0xef30c340bfb3d1c0 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403110:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada60a17c00 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.403113:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17c00/0xef30c340bfb3d1c0 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403119:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17c00/0xef30c340bfb3d1c0 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403125:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17c00/0xef30c340bfb3d1c0 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403131:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.403133:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17c00/0xef30c340bfb3d1c0 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403139:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17c00/0xef30c340bfb3d1c0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403152:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16a00/0xef30c340bfb3d1c7 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403159:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16a00/0xef30c340bfb3d1c7 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403165:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16a00/0xef30c340bfb3d1c7 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403172:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16a00/0xef30c340bfb3d1c7 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403177:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16a00/0xef30c340bfb3d1c7 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403183:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada60a16a00 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.403187:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16a00/0xef30c340bfb3d1c7 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403193:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16a00/0xef30c340bfb3d1c7 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403200:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16a00/0xef30c340bfb3d1c7 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403205:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.403208:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16a00/0xef30c340bfb3d1c7 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403214:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16a00/0xef30c340bfb3d1c7 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403228:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17000/0xef30c340bfb3d1ce lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403235:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17000/0xef30c340bfb3d1ce lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403241:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17000/0xef30c340bfb3d1ce lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403247:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17000/0xef30c340bfb3d1ce lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403252:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17000/0xef30c340bfb3d1ce lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403259:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada60a17000 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.403263:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17000/0xef30c340bfb3d1ce lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 14 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403268:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17000/0xef30c340bfb3d1ce lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 14 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403274:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17000/0xef30c340bfb3d1ce lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 14 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403280:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.403283:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17000/0xef30c340bfb3d1ce lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 14 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403289:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17000/0xef30c340bfb3d1ce lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 14 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403301:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16600/0xef30c340bfb3d1d5 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 13 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403308:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16600/0xef30c340bfb3d1d5 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 13 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403313:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16600/0xef30c340bfb3d1d5 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 13 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403320:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16600/0xef30c340bfb3d1d5 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 13 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403326:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16600/0xef30c340bfb3d1d5 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 13 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403332:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada60a16600 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.403334:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16600/0xef30c340bfb3d1d5 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 13 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403340:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16600/0xef30c340bfb3d1d5 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 13 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403346:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16600/0xef30c340bfb3d1d5 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 13 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403351:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.403354:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16600/0xef30c340bfb3d1d5 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 13 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403361:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16600/0xef30c340bfb3d1d5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 13 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403373:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17200/0xef30c340bfb3d1dc lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403379:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17200/0xef30c340bfb3d1dc lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403385:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17200/0xef30c340bfb3d1dc lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403391:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17200/0xef30c340bfb3d1dc lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403397:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17200/0xef30c340bfb3d1dc lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403402:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada60a17200 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.403405:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17200/0xef30c340bfb3d1dc lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 12 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403411:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17200/0xef30c340bfb3d1dc lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 12 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403416:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17200/0xef30c340bfb3d1dc lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 12 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403422:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.403425:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17200/0xef30c340bfb3d1dc lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 12 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403431:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17200/0xef30c340bfb3d1dc lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 12 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403443:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16e00/0xef30c340bfb3d1e3 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403448:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16e00/0xef30c340bfb3d1e3 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403454:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16e00/0xef30c340bfb3d1e3 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403460:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16e00/0xef30c340bfb3d1e3 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403466:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16e00/0xef30c340bfb3d1e3 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403471:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada60a16e00 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.403473:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16e00/0xef30c340bfb3d1e3 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403480:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16e00/0xef30c340bfb3d1e3 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403486:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16e00/0xef30c340bfb3d1e3 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403490:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.403494:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16e00/0xef30c340bfb3d1e3 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403500:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16e00/0xef30c340bfb3d1e3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403513:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16000/0xef30c340bfb3d1ea lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 10 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403519:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16000/0xef30c340bfb3d1ea lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 10 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403525:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16000/0xef30c340bfb3d1ea lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 10 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403532:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16000/0xef30c340bfb3d1ea lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 10 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403537:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16000/0xef30c340bfb3d1ea lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 10 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403543:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada60a16000 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.403546:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16000/0xef30c340bfb3d1ea lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 10 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403551:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16000/0xef30c340bfb3d1ea lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 10 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403558:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16000/0xef30c340bfb3d1ea lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 10 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403564:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.403567:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16000/0xef30c340bfb3d1ea lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 10 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403573:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16000/0xef30c340bfb3d1ea lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 10 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403585:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17800/0xef30c340bfb3d1f1 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 9 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403591:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17800/0xef30c340bfb3d1f1 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 9 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403597:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17800/0xef30c340bfb3d1f1 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 9 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403604:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17800/0xef30c340bfb3d1f1 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 9 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403610:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17800/0xef30c340bfb3d1f1 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 9 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403633:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada60a17800 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.403636:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17800/0xef30c340bfb3d1f1 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 9 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403642:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17800/0xef30c340bfb3d1f1 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 9 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403648:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17800/0xef30c340bfb3d1f1 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 9 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403654:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.403657:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17800/0xef30c340bfb3d1f1 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 9 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403664:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17800/0xef30c340bfb3d1f1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 9 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403677:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17a00/0xef30c340bfb3d1f8 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403683:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17a00/0xef30c340bfb3d1f8 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403688:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17a00/0xef30c340bfb3d1f8 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403693:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17a00/0xef30c340bfb3d1f8 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403699:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17a00/0xef30c340bfb3d1f8 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403707:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada60a17a00 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.403710:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17a00/0xef30c340bfb3d1f8 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 8 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403716:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17a00/0xef30c340bfb3d1f8 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 8 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403722:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17a00/0xef30c340bfb3d1f8 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 8 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403727:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.403730:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17a00/0xef30c340bfb3d1f8 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 8 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403735:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17a00/0xef30c340bfb3d1f8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 8 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403747:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16c00/0xef30c340bfb3d1ff lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403752:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16c00/0xef30c340bfb3d1ff lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403759:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16c00/0xef30c340bfb3d1ff lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403766:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16c00/0xef30c340bfb3d1ff lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403773:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16c00/0xef30c340bfb3d1ff lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403783:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada60a16c00 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.403787:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16c00/0xef30c340bfb3d1ff lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 7 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403793:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16c00/0xef30c340bfb3d1ff lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 7 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403800:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16c00/0xef30c340bfb3d1ff lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 7 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403806:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.403810:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16c00/0xef30c340bfb3d1ff lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 7 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403816:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16c00/0xef30c340bfb3d1ff lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 7 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403832:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17400/0xef30c340bfb3d206 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403839:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17400/0xef30c340bfb3d206 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403846:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17400/0xef30c340bfb3d206 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403890:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17400/0xef30c340bfb3d206 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403897:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17400/0xef30c340bfb3d206 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403907:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada60a17400 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.403910:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17400/0xef30c340bfb3d206 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 6 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403917:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17400/0xef30c340bfb3d206 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 6 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403923:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17400/0xef30c340bfb3d206 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 6 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403929:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.403933:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17400/0xef30c340bfb3d206 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 6 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403939:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17400/0xef30c340bfb3d206 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 6 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403955:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17e00/0xef30c340bfb3d20d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403961:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17e00/0xef30c340bfb3d20d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403967:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17e00/0xef30c340bfb3d20d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403974:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17e00/0xef30c340bfb3d20d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403980:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17e00/0xef30c340bfb3d20d lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403989:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada60a17e00 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.403992:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17e00/0xef30c340bfb3d20d lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 5 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.403997:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17e00/0xef30c340bfb3d20d lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 5 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.404003:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17e00/0xef30c340bfb3d20d lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 5 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.404010:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.404013:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17e00/0xef30c340bfb3d20d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 5 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.404018:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17e00/0xef30c340bfb3d20d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 5 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.404033:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c000/0xef30c340bfb3d214 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.404039:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c000/0xef30c340bfb3d214 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.404045:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c000/0xef30c340bfb3d214 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.404051:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c000/0xef30c340bfb3d214 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.404057:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c000/0xef30c340bfb3d214 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.404067:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada5172c000 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.404070:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c000/0xef30c340bfb3d214 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.404076:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c000/0xef30c340bfb3d214 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.404083:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c000/0xef30c340bfb3d214 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.404088:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.404092:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c000/0xef30c340bfb3d214 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.404098:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c000/0xef30c340bfb3d214 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.404114:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d000/0xef30c340bfb3d21b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.404121:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d000/0xef30c340bfb3d21b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.404126:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d000/0xef30c340bfb3d21b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.404132:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d000/0xef30c340bfb3d21b lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.404138:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d000/0xef30c340bfb3d21b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.404147:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada5172d000 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.404150:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d000/0xef30c340bfb3d21b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 3 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.404157:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d000/0xef30c340bfb3d21b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 3 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.404164:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d000/0xef30c340bfb3d21b lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 3 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.404170:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.404173:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d000/0xef30c340bfb3d21b lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.404179:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d000/0xef30c340bfb3d21b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.404195:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d400/0xef30c340bfb3d222 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.404203:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d400/0xef30c340bfb3d222 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.404209:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d400/0xef30c340bfb3d222 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.404217:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d400/0xef30c340bfb3d222 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.404223:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d400/0xef30c340bfb3d222 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.404233:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada5172d400 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571772.404236:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d400/0xef30c340bfb3d222 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 2 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.404242:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d400/0xef30c340bfb3d222 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 2 type: IBT flags: 0x206400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.404247:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d400/0xef30c340bfb3d222 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 2 type: IBT flags: 0x20e400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.404252:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571772.404255:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d400/0xef30c340bfb3d222 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571772.404261:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d400/0xef30c340bfb3d222 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571772.404303:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a34700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613564544:192.168.202.113@tcp:63:ll_sa.0 00000100:00100000:0.0:1788571772.432836:0:78193:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f2d80 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:78193:1875449613564416:192.168.202.113@tcp:36:rm.0 00000080:00200000:0.0:1788571772.432967:0:78193:0:(vvp_io.c:1826:vvp_io_init()) [0x200015bc1:0xa:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1788571772.432984:0:78193:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1788571772.433024:0:78193:0:(vvp_io.c:304:vvp_io_fini()) [0x200015bc1:0xa:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1788571772.433034:0:78193:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200015bc1:0xa:0x0](ffff9ada5825f448) 00000008:00010000:0.0:1788571772.433081:0:78193:0:(osc_object.c:202:osc_object_ast_clear()) ### update lock size 0 blocks 0 [cma]time: 0 0 0 by oinfo size 0 blocks 0 [cma]time 0 0 0 ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 000000002bdeef89/0xef30c340bfb3d04d lrc: 1/0,0 mode: PR/PR res: [0x280000401:0x5e6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0xdb7edb52b82f68d9 expref: -99 pid: 78195 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000100:00100000:2.0:1788571776.384743:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613564800:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571776.384836:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613565056:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571776.384890:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:2.0:1788571776.384897:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a34700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449613564672:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571776.384930:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613564928:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571776.384950:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613565184:192.168.202.113@tcp:400:kworker.0 00010000:00010000:2.0:1788571776.887443:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788571777.564364:0:2378:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.113@tcp (source: 12345-192.168.202.113@tcp) 00000100:00100000:2.0:1788571777.564389:0:2378:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.113@tcp (source: 12345-192.168.202.113@tcp) 00000100:00100000:2.0:1788571777.564411:0:2378:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.113@tcp (source: 12345-192.168.202.113@tcp) 00000100:00100000:2.0:1788571777.572904:0:2377:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.113@tcp (source: 12345-192.168.202.113@tcp) 00000100:00100000:2.0:1788571777.573014:0:2377:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.113@tcp (source: 12345-192.168.202.113@tcp) 00000100:00100000:2.0:1788571777.573026:0:2377:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.113@tcp (source: 12345-192.168.202.113@tcp) 00000100:00100000:2.0:1788571777.573055:0:2377:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.113@tcp (source: 12345-192.168.202.113@tcp) 00000100:00100000:2.0:1788571777.579998:0:33410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875449638480128 00000100:00100000:2.0:1788571777.580012:0:33410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875449638480128 00000100:00100000:2.0:1788571777.580056:0:33410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875449638480256 00000100:00100000:2.0:1788571777.580059:0:33410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875449638480256 00000100:00100000:2.0:1788571777.580071:0:33410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875449638480384 00000100:00100000:2.0:1788571777.580074:0:33410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875449638480384 00000100:00100000:2.0:1788571777.580084:0:33410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875449638480640 00000100:00100000:2.0:1788571777.580086:0:33410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875449638480640 00000100:00100000:2.0:1788571777.580104:0:33410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.113@tcp, seq: 223 00000100:00100000:2.0:1788571777.580114:0:33410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9ada4653c450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_002:LOV_OSC_UUID+5:43965:x1875449638479616:12345-192.168.202.113@tcp:104: 00010000:00010000:2.0:1788571777.580131:0:33410:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 0000000078fe1a46/0xef30c340bfb3d03f lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x5e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x420000010000 nid: local remote: 0xdb7edb52b82f68cb expref: -99 pid: 78195 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571777.580234:0:33410:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to priority blp list, 2 blwis in pool 00000100:00100000:2.0:1788571777.580242:0:33410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9ada4653c450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_002:LOV_OSC_UUID+5:43965:x1875449638479616:12345-192.168.202.113@tcp:104: Request processed in 128us (15828us total) trans 0 rc 0/0 00000100:00100000:2.0:1788571777.580249:0:33410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.113@tcp, seq: 223 00000100:00100000:2.0:1788571777.580268:0:33410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.113@tcp, seq: 224 00000100:00100000:2.0:1788571777.580276:0:33410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9ada60a36300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_002:LOV_OSC_UUID+7:17011:x1875449638480128:12345-192.168.202.113@tcp:104: 00010000:00010000:2.0:1788571777.580302:0:33410:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 00000000348f9350/0xef30c340bfb3cfe4 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x5e2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x420000010000 nid: local remote: 0xdb7edb52b82f679e expref: -99 pid: 78193 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571777.580341:0:33410:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to priority blp list, 3 blwis in pool 00000100:00100000:2.0:1788571777.580347:0:33410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9ada60a36300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_002:LOV_OSC_UUID+7:17011:x1875449638480128:12345-192.168.202.113@tcp:104: Request processed in 71us (7453us total) trans 0 rc 0/0 00000100:00100000:2.0:1788571777.580353:0:33410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.113@tcp, seq: 224 00000100:00100000:2.0:1788571777.580369:0:33410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.113@tcp, seq: 225 00000100:00100000:2.0:1788571777.580375:0:33410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9ada60a36680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_002:LOV_OSC_UUID+6:43967:x1875449638480256:12345-192.168.202.113@tcp:104: 00010000:00010000:2.0:1788571777.580384:0:33410:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 000000007186fa62/0xef30c340bfb3d000 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x5e3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x420000010000 nid: local remote: 0xdb7edb52b82f6800 expref: -99 pid: 78193 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571777.580409:0:33410:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to priority blp list, 4 blwis in pool 00000100:00100000:2.0:1788571777.580414:0:33410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9ada60a36680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_002:LOV_OSC_UUID+6:43967:x1875449638480256:12345-192.168.202.113@tcp:104: Request processed in 40us (7401us total) trans 0 rc 0/0 00000100:00100000:2.0:1788571777.580420:0:33410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.113@tcp, seq: 225 00000100:00100000:2.0:1788571777.580432:0:33410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.113@tcp, seq: 226 00000100:00100000:2.0:1788571777.580438:0:33410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9ada60a37480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_002:LOV_OSC_UUID+5:43966:x1875449638480384:12345-192.168.202.113@tcp:104: 00010000:00010000:2.0:1788571777.580445:0:33410:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 00000000424f9e68/0xef30c340bfb3d01c lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x5e4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x420000010000 nid: local remote: 0xdb7edb52b82f6862 expref: -99 pid: 78193 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571777.580467:0:33410:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/5 locks to priority blp list, 5 blwis in pool 00000100:00100000:2.0:1788571777.580471:0:33410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9ada60a37480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_002:LOV_OSC_UUID+5:43966:x1875449638480384:12345-192.168.202.113@tcp:104: Request processed in 34us (7445us total) trans 0 rc 0/0 00000100:00100000:2.0:1788571777.580477:0:33410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.113@tcp, seq: 226 00000100:00100000:2.0:1788571777.580488:0:33410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.113@tcp, seq: 227 00000100:00100000:2.0:1788571777.580494:0:33410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9ada60a34000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_002:LOV_OSC_UUID+4:43968:x1875449638480640:12345-192.168.202.113@tcp:104: 00010000:00010000:2.0:1788571777.580501:0:33410:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 0000000005bd4a7e/0xef30c340bfb3d038 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x5e5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x420000010000 nid: local remote: 0xdb7edb52b82f68c4 expref: -99 pid: 78193 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571777.581148:0:33410:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/6 locks to priority blp list, 6 blwis in pool 00000100:00100000:2.0:1788571777.581159:0:33410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9ada60a34000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_002:LOV_OSC_UUID+4:43968:x1875449638480640:12345-192.168.202.113@tcp:104: Request processed in 663us (8101us total) trans 0 rc 0/0 00000100:00100000:2.0:1788571777.581166:0:33410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.113@tcp, seq: 227 00010000:00010000:2.0:1788571777.588132:0:15205:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 000000002bdeef89/0xef30c340bfb3d04d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0xdb7edb52b82f68d9 expref: -99 pid: 78195 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571777.588153:0:15205:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1788571777.588190:0:15205:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 000000002bdeef89/0xef30c340bfb3d04d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0xdb7edb52b82f68d9 expref: -99 pid: 78195 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571777.588199:0:15205:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 000000002bdeef89/0xef30c340bfb3d04d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x5e6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0xdb7edb52b82f68d9 expref: -99 pid: 78195 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571777.588230:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571777.588232:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571777.588236:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571778.939271:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788571781.496191:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a49c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449613566592:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571781.496527:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1788571781.496536:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a48000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613566720:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571781.502232:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a48e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613566848:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571781.508574:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1788571781.508605:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a48380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613566976:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571781.509784:0:2386:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 193273528350 gen 5 00000100:00100000:2.0:1788571781.509943:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a48000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613566720:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571781.509959:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a49c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449613566592:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571781.530451:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a49880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613567104:192.168.202.113@tcp:400:kworker.0 00000001:02000400:2.0:1788571781.909243:0:78760:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == replay-single test 33b: test fid seq allocation ======= 21:29:39 (1788571779) 00010000:00010000:2.0:1788571782.011070:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788571783.429627:0:78780:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada44fdfe40 going to sleep for 11 seconds 00000100:00100000:0.0:1788571783.429712:0:78780:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a48380 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78780:1875449613567744:192.168.202.113@tcp:35:createmany.0 00000002:00100000:0.0:1788571783.429755:0:78780:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9ada60a48e00 x1875449613567488/t193273528351(193273528351) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571799 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1788571783.431170:0:78780:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571783.431205:0:78780:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571783.431283:0:78780:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617c00/0xef30c340bfb3d237 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78780 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.431296:0:78780:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617c00/0xef30c340bfb3d237 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78780 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.431308:0:78780:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617c00/0xef30c340bfb3d237 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78780 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571783.431339:0:78780:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f1180 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78780:1875449613567872:192.168.202.113@tcp:101:createmany.0 00000100:00100000:0.0:1788571783.431591:0:78780:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada609d6180 going to sleep for 11 seconds 00000100:00100000:0.0:1788571783.437331:0:78780:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f1180 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78780:1875449613567872:192.168.202.113@tcp:101:createmany.0 00010000:00010000:0.0:1788571783.437367:0:78780:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617c00/0xef30c340bfb3d237 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6a4c expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.437381:0:78780:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617c00/0xef30c340bfb3d237 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6a4c expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.437405:0:78780:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617c00/0xef30c340bfb3d237 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6a4c expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.437419:0:78780:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617c00/0xef30c340bfb3d237 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6a4c expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.437434:0:78780:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617c00/0xef30c340bfb3d237 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6a4c expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.437444:0:78780:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617c00/0xef30c340bfb3d237 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6a4c expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.437453:0:78780:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617c00/0xef30c340bfb3d237 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6a4c expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788571783.437467:0:78780:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada516f1180 x1875449613567872/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788571799 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788571783.437496:0:78780:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617c00/0xef30c340bfb3d237 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6a4c expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.437513:0:78780:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1788571783.437521:0:78780:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1788571783.437558:0:78780:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:0.0:1788571783.437566:0:78780:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d237 00000080:00010000:0.0:1788571783.437572:0:78780:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571783.437576:0:78780:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d237 from it ffffab0ec145bbf8 00010000:00010000:0.0:1788571783.437580:0:78780:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617c00/0xef30c340bfb3d237 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6a4c expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.437591:0:78780:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617c00/0xef30c340bfb3d237 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6a4c expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571783.437622:0:78780:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:0.0:1788571783.437654:0:78780:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571783.437673:0:78780:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571783.437685:0:78780:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617c00/0xef30c340bfb3d237 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6a4c expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571783.437697:0:78780:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617c00/0xef30c340bfb3d237 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6a4c expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571783.437705:0:78780:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788571783.437709:0:78780:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571783.437713:0:78780:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d237 00000080:00010000:0.0:1788571783.437717:0:78780:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d237 from it ffffab0ec145bb38 00010000:00010000:0.0:1788571783.437720:0:78780:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617c00/0xef30c340bfb3d237 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6a4c expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571783.437728:0:78780:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617c00/0xef30c340bfb3d237 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6a4c expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788571783.437736:0:78780:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:0.0:1788571783.437742:0:78780:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33b.replay-single-1, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), file ffff9ada4676fb00, open_flags 8042, mode 81a4 opened 0 00000080:00200000:0.0:1788571783.437752:0:78780:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33b.replay-single-1, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000002:00010000:0.0:1788571783.437787:0:78780:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33b.replay-single-1,[0x200015bc1:0xc:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:0.0:1788571783.437811:0:78780:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617c00/0xef30c340bfb3d237 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b82f6a4c expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571783.437899:0:78780:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617c00/0xef30c340bfb3d237 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f6a4c expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571783.437907:0:78780:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1788571783.437910:0:78780:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617c00/0xef30c340bfb3d237 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f6a4c expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571783.437950:0:78780:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617a00/0xef30c340bfb3d23e lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78780 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.437959:0:78780:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617a00/0xef30c340bfb3d23e lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78780 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:0.0:1788571783.437971:0:78780:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:0.0:1788571783.437979:0:78780:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada516f1180 x1875449613568128/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:0.0:1788571783.437989:0:78780:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617a00/0xef30c340bfb3d23e lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78780 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571783.438013:0:78780:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f1180 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78780:1875449613568128:192.168.202.113@tcp:101:createmany.0 00000100:00100000:0.0:1788571783.438205:0:78780:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada609d6180 going to sleep for 11 seconds 00000100:00100000:0.0:1788571783.444263:0:78780:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f1180 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78780:1875449613568128:192.168.202.113@tcp:101:createmany.0 00010000:00010000:0.0:1788571783.444300:0:78780:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617a00/0xef30c340bfb3d23e lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6a68 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.444338:0:78780:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617a00/0xef30c340bfb3d23e lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0xc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6a68 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.444348:0:78780:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617a00/0xef30c340bfb3d23e lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6a68 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.444361:0:78780:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617a00/0xef30c340bfb3d23e lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6a68 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.444380:0:78780:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617a00/0xef30c340bfb3d23e lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6a68 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.444407:0:78780:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617a00/0xef30c340bfb3d23e lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6a68 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.444417:0:78780:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617a00/0xef30c340bfb3d23e lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6a68 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.444425:0:78780:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617a00/0xef30c340bfb3d23e lrc: 4/1,0 mode: PR/PR res: [0x200015bc1:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6a68 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788571783.444440:0:78780:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9ada516f1180 x1875449613568128/t193273528353(193273528353) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/664 e 0 to 0 dl 1788571799 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:0.0:1788571783.444455:0:78780:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada516f1180 x1875449613568128/t193273528353(193273528353) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/664 e 0 to 0 dl 1788571799 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788571783.444472:0:78780:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617a00/0xef30c340bfb3d23e lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6a68 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:0.0:1788571783.444500:0:78780:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617a00/0xef30c340bfb3d23e lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6a68 expref: -99 pid: 78780 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.444512:0:78780:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8590023617/12 (0 0) 00000080:00200000:0.0:1788571783.444519:0:78780:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1788571783.444538:0:78780:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33b.replay-single-1, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000080:00200000:0.0:1788571783.444578:0:78780:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200015bc1:0xc:0x0](ffff9ada5825f448) 00000080:00200000:0.0:1788571783.444676:0:78780:0:(file.c:6843:ll_layout_conf()) [0x200015bc1:0xc:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:0.0:1788571783.444682:0:78780:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200015bc1:0xc:0x0](ffff9ada5825f448) 00000080:00010000:0.0:1788571783.444685:0:78780:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0xc:0x0](ffff9ada5825f448) for lock 0xef30c340bfb3d23e 00000080:00010000:0.0:1788571783.444697:0:78780:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0xc:0x0](ffff9ada5825f448) 00000080:00200008:0.0:1788571783.444718:0:78780:0:(file.c:1090:ll_file_open()) START file f33b.replay-single-1:[0x200015bc1:0xc:0x0](ffff9ada5825f448), flags 100102 00000002:00100000:0.0:1788571783.444744:0:78780:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada516f1180 x1875449613568128/t193273528353(193273528353) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571799 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:0.0:1788571783.444798:0:78780:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d23e from it ffff9ada517618a0 00010000:00010000:0.0:1788571783.444803:0:78780:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617a00/0xef30c340bfb3d23e lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f6a68 expref: -99 pid: 78780 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.444813:0:78780:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617a00/0xef30c340bfb3d23e lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f6a68 expref: -99 pid: 78780 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571783.444842:0:78780:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200015bc1:0xc:0x0](ffff9ada5825f448), inode mode 81a4 mask 40 00000080:00200008:2.0:1788571783.445279:0:78780:0:(file.c:479:ll_file_release()) START file f33b.replay-single-1:[0x200015bc1:0xc:0x0](ffff9ada5825f448), flags 100002 00000002:00100000:2.0:1788571783.445336:0:78780:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9ada516f1180 x1875449613568128/t193273528353(193273528353) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571799 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1788571783.445349:0:78780:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1788571783.445355:0:78780:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada60a34380 x1875449613568384/t0(0) o35->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1788571783.445380:0:78780:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a34380 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78780:1875449613568384:192.168.202.113@tcp:35:createmany.0 00000100:00100000:2.0:1788571783.445565:0:78780:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada607ff600 going to sleep for 11 seconds 00000080:00200000:0.0:1788571783.486215:0:78780:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:0.0:1788571783.486244:0:78780:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d245 00000080:00010000:0.0:1788571783.486255:0:78780:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571783.486258:0:78780:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d245 from it ffffab0ec145bbf8 00010000:00010000:0.0:1788571783.486266:0:78780:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759c00/0xef30c340bfb3d245 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6a7d expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.486283:0:78780:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759c00/0xef30c340bfb3d245 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6a7d expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571783.486323:0:78780:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:0.0:1788571783.486357:0:78780:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571783.486380:0:78780:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571783.486409:0:78780:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759c00/0xef30c340bfb3d245 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6a7d expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571783.486424:0:78780:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759c00/0xef30c340bfb3d245 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6a7d expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571783.486434:0:78780:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788571783.486442:0:78780:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571783.486445:0:78780:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d245 00000080:00010000:0.0:1788571783.486449:0:78780:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d245 from it ffffab0ec145bb38 00010000:00010000:0.0:1788571783.486453:0:78780:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759c00/0xef30c340bfb3d245 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6a7d expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571783.486460:0:78780:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759c00/0xef30c340bfb3d245 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6a7d expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788571783.486469:0:78780:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:0.0:1788571783.486475:0:78780:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33b.replay-single-2, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), file ffff9ada6169f980, open_flags 8042, mode 81a4 opened 0 00000080:00200000:0.0:1788571783.486485:0:78780:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33b.replay-single-2, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000002:00010000:0.0:1788571783.486510:0:78780:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33b.replay-single-2,[0x200015bc1:0xd:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:0.0:1788571783.486525:0:78780:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759c00/0xef30c340bfb3d245 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b82f6a7d expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571783.486596:0:78780:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759c00/0xef30c340bfb3d245 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f6a7d expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571783.486609:0:78780:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1788571783.486612:0:78780:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759c00/0xef30c340bfb3d245 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f6a7d expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571783.486661:0:78780:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617c00/0xef30c340bfb3d24c lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78780 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.486670:0:78780:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617c00/0xef30c340bfb3d24c lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78780 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:0.0:1788571783.486683:0:78780:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:0.0:1788571783.486698:0:78780:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada516f1880 x1875449613568768/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:0.0:1788571783.486708:0:78780:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617c00/0xef30c340bfb3d24c lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78780 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571783.486735:0:78780:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f1880 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78780:1875449613568768:192.168.202.113@tcp:101:createmany.0 00000100:00100000:0.0:1788571783.486987:0:78780:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada609d6f00 going to sleep for 11 seconds 00000100:00100000:0.0:1788571783.494243:0:78780:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f1880 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78780:1875449613568768:192.168.202.113@tcp:101:createmany.0 00010000:00010000:0.0:1788571783.494280:0:78780:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617c00/0xef30c340bfb3d24c lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6a99 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.494314:0:78780:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617c00/0xef30c340bfb3d24c lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0xd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6a99 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.494323:0:78780:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617c00/0xef30c340bfb3d24c lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6a99 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.494342:0:78780:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617c00/0xef30c340bfb3d24c lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6a99 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.494354:0:78780:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617c00/0xef30c340bfb3d24c lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6a99 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.494369:0:78780:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617c00/0xef30c340bfb3d24c lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6a99 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.494377:0:78780:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617c00/0xef30c340bfb3d24c lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6a99 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.494384:0:78780:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617c00/0xef30c340bfb3d24c lrc: 4/1,0 mode: PR/PR res: [0x200015bc1:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6a99 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788571783.494409:0:78780:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9ada516f1880 x1875449613568768/t193273528355(193273528355) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/664 e 0 to 0 dl 1788571799 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:0.0:1788571783.494424:0:78780:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada516f1880 x1875449613568768/t193273528355(193273528355) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/664 e 0 to 0 dl 1788571799 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788571783.494441:0:78780:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617c00/0xef30c340bfb3d24c lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6a99 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:0.0:1788571783.494457:0:78780:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617c00/0xef30c340bfb3d24c lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6a99 expref: -99 pid: 78780 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.494469:0:78780:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8590023617/13 (0 0) 00000080:00200000:0.0:1788571783.494490:0:78780:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1788571783.494507:0:78780:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33b.replay-single-2, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000080:00200000:0.0:1788571783.494559:0:78780:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200015bc1:0xd:0x0](ffff9ada5825cb08) 00000080:00200000:0.0:1788571783.494662:0:78780:0:(file.c:6843:ll_layout_conf()) [0x200015bc1:0xd:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:0.0:1788571783.494668:0:78780:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200015bc1:0xd:0x0](ffff9ada5825cb08) 00000080:00010000:0.0:1788571783.494672:0:78780:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0xd:0x0](ffff9ada5825cb08) for lock 0xef30c340bfb3d24c 00000080:00010000:0.0:1788571783.494684:0:78780:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0xd:0x0](ffff9ada5825cb08) 00000080:00200008:0.0:1788571783.494706:0:78780:0:(file.c:1090:ll_file_open()) START file f33b.replay-single-2:[0x200015bc1:0xd:0x0](ffff9ada5825cb08), flags 100102 00000002:00100000:0.0:1788571783.494733:0:78780:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada516f1880 x1875449613568768/t193273528355(193273528355) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571799 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:0.0:1788571783.494789:0:78780:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d24c from it ffff9ada517618a0 00010000:00010000:0.0:1788571783.494794:0:78780:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617c00/0xef30c340bfb3d24c lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f6a99 expref: -99 pid: 78780 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.494806:0:78780:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617c00/0xef30c340bfb3d24c lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f6a99 expref: -99 pid: 78780 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571783.494834:0:78780:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200015bc1:0xd:0x0](ffff9ada5825cb08), inode mode 81a4 mask 40 00000080:00200008:0.0:1788571783.494918:0:78780:0:(file.c:479:ll_file_release()) START file f33b.replay-single-2:[0x200015bc1:0xd:0x0](ffff9ada5825cb08), flags 100002 00000002:00100000:0.0:1788571783.494974:0:78780:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9ada516f1880 x1875449613568768/t193273528355(193273528355) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571799 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:0.0:1788571783.494986:0:78780:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:0.0:1788571783.494991:0:78780:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada516f1c00 x1875449613569024/t0(0) o35->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:0.0:1788571783.495018:0:78780:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f1c00 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78780:1875449613569024:192.168.202.113@tcp:35:createmany.0 00000100:00100000:0.0:1788571783.495123:0:78780:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada609d6600 going to sleep for 11 seconds 00000100:00100000:0.0:1788571783.499183:0:78780:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f1c00 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78780:1875449613569024:192.168.202.113@tcp:35:createmany.0 00000002:00100000:0.0:1788571783.499233:0:78780:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9ada516f1880 x1875449613568768/t193273528355(193273528355) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571799 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1788571783.499341:0:78780:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571783.499366:0:78780:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571783.499456:0:78780:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d253 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78780 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.499468:0:78780:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d253 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78780 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.499494:0:78780:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d253 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78780 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571783.499520:0:78780:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0380 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78780:1875449613569152:192.168.202.113@tcp:101:createmany.0 00000100:00100000:0.0:1788571783.499722:0:78780:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada609d6600 going to sleep for 11 seconds 00000100:00100000:0.0:1788571783.525029:0:78780:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0380 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78780:1875449613569152:192.168.202.113@tcp:101:createmany.0 00010000:00010000:0.0:1788571783.525070:0:78780:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d253 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6aae expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.525085:0:78780:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d253 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6aae expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.525101:0:78780:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d253 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6aae expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.525115:0:78780:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d253 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6aae expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.525132:0:78780:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d253 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6aae expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.525141:0:78780:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d253 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6aae expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.525150:0:78780:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d253 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6aae expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788571783.525164:0:78780:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada516f0380 x1875449613569152/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788571799 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788571783.525180:0:78780:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d253 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6aae expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.525196:0:78780:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1788571783.525204:0:78780:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1788571783.525242:0:78780:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:0.0:1788571783.525249:0:78780:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d253 00000080:00010000:0.0:1788571783.525256:0:78780:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571783.525261:0:78780:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d253 from it ffffab0ec145bbf8 00010000:00010000:0.0:1788571783.525265:0:78780:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d253 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6aae expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.525275:0:78780:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d253 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6aae expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571783.525311:0:78780:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:0.0:1788571783.525343:0:78780:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571783.525361:0:78780:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571783.525371:0:78780:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d253 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6aae expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571783.525384:0:78780:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d253 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6aae expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571783.525391:0:78780:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788571783.525461:0:78780:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571783.525466:0:78780:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d253 00000080:00010000:0.0:1788571783.525471:0:78780:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d253 from it ffffab0ec145bb38 00010000:00010000:0.0:1788571783.525474:0:78780:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d253 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6aae expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571783.525483:0:78780:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d253 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6aae expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788571783.525493:0:78780:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:0.0:1788571783.525499:0:78780:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33b.replay-single-3, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), file ffff9ada4676f500, open_flags 8042, mode 81a4 opened 0 00000080:00200000:0.0:1788571783.525510:0:78780:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33b.replay-single-3, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000002:00010000:0.0:1788571783.525536:0:78780:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33b.replay-single-3,[0x200015bc1:0xe:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:0.0:1788571783.525550:0:78780:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d253 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b82f6aae expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571783.525612:0:78780:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d253 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f6aae expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571783.525620:0:78780:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1788571783.525623:0:78780:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d253 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f6aae expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571783.525667:0:78780:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617800/0xef30c340bfb3d25a lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78780 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.525675:0:78780:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617800/0xef30c340bfb3d25a lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78780 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:0.0:1788571783.525687:0:78780:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:0.0:1788571783.525695:0:78780:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada516f0380 x1875449613569408/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:0.0:1788571783.525705:0:78780:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617800/0xef30c340bfb3d25a lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78780 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571783.525729:0:78780:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0380 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78780:1875449613569408:192.168.202.113@tcp:101:createmany.0 00000100:00100000:2.0:1788571783.601959:0:78780:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada5051ff00 going to sleep for 11 seconds 00000100:00100000:0.0:1788571783.623118:0:78780:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f6a00 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78780:1875449613570304:192.168.202.113@tcp:35:createmany.0 00000002:00100000:0.0:1788571783.623166:0:78780:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9ada603f6300 x1875449613570048/t193273528359(193273528359) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571799 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1788571783.623273:0:78780:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571783.623299:0:78780:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571783.623374:0:78780:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d26f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78780 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.623387:0:78780:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d26f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78780 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.623411:0:78780:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d26f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78780 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571783.623437:0:78780:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f1500 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78780:1875449613570432:192.168.202.113@tcp:101:createmany.0 00000100:00100000:0.0:1788571783.623540:0:78780:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada609d6600 going to sleep for 11 seconds 00000100:00100000:0.0:1788571783.648641:0:78780:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f1500 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78780:1875449613570432:192.168.202.113@tcp:101:createmany.0 00010000:00010000:0.0:1788571783.648678:0:78780:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d26f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6b10 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.648691:0:78780:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d26f lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6b10 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.648704:0:78780:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d26f lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6b10 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.648716:0:78780:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d26f lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6b10 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.648731:0:78780:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d26f lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6b10 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.648739:0:78780:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d26f lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6b10 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.648753:0:78780:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d26f lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6b10 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788571783.648766:0:78780:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada516f1500 x1875449613570432/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788571799 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788571783.648794:0:78780:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d26f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6b10 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.648816:0:78780:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1788571783.648824:0:78780:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1788571783.648899:0:78780:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:0.0:1788571783.648907:0:78780:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d26f 00000080:00010000:0.0:1788571783.648913:0:78780:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571783.648917:0:78780:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d26f from it ffffab0ec145bbf8 00010000:00010000:0.0:1788571783.648921:0:78780:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d26f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6b10 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.648931:0:78780:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d26f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6b10 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571783.648959:0:78780:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:0.0:1788571783.648992:0:78780:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571783.649012:0:78780:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571783.649022:0:78780:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d26f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6b10 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571783.649039:0:78780:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d26f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6b10 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571783.649046:0:78780:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788571783.649051:0:78780:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571783.649053:0:78780:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d26f 00000080:00010000:0.0:1788571783.649058:0:78780:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d26f from it ffffab0ec145bb38 00010000:00010000:0.0:1788571783.649060:0:78780:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d26f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6b10 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571783.649067:0:78780:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d26f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6b10 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788571783.649075:0:78780:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:0.0:1788571783.649081:0:78780:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33b.replay-single-5, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), file ffff9ada4676ef00, open_flags 8042, mode 81a4 opened 0 00000080:00200000:0.0:1788571783.649091:0:78780:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33b.replay-single-5, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000002:00010000:0.0:1788571783.649115:0:78780:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33b.replay-single-5,[0x200015bc1:0x10:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:0.0:1788571783.649128:0:78780:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d26f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b82f6b10 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571783.649184:0:78780:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d26f lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f6b10 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571783.649191:0:78780:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1788571783.649194:0:78780:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d26f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f6b10 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571783.649243:0:78780:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d276 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78780 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.649250:0:78780:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d276 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78780 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:0.0:1788571783.649263:0:78780:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:0.0:1788571783.649271:0:78780:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada516f1500 x1875449613570688/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:0.0:1788571783.649281:0:78780:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d276 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78780 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571783.649305:0:78780:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f1500 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78780:1875449613570688:192.168.202.113@tcp:101:createmany.0 00000100:00100000:2.0:1788571783.653851:0:78780:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada609d6600 going to sleep for 11 seconds 00000100:00100000:0.0:1788571783.676046:0:78780:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f1500 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78780:1875449613570688:192.168.202.113@tcp:101:createmany.0 00010000:00010000:0.0:1788571783.676083:0:78780:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d276 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6b2c expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.676119:0:78780:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d276 lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x10:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6b2c expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.676129:0:78780:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d276 lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6b2c expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.676141:0:78780:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d276 lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6b2c expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.676160:0:78780:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d276 lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6b2c expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.676177:0:78780:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d276 lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6b2c expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.676186:0:78780:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d276 lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6b2c expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.676195:0:78780:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d276 lrc: 4/1,0 mode: PR/PR res: [0x200015bc1:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6b2c expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788571783.676210:0:78780:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9ada516f1500 x1875449613570688/t193273528361(193273528361) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/664 e 0 to 0 dl 1788571799 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:0.0:1788571783.676226:0:78780:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada516f1500 x1875449613570688/t193273528361(193273528361) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/664 e 0 to 0 dl 1788571799 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788571783.676243:0:78780:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d276 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6b2c expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:0.0:1788571783.676259:0:78780:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d276 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6b2c expref: -99 pid: 78780 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.676271:0:78780:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8590023617/16 (0 0) 00000080:00200000:0.0:1788571783.676279:0:78780:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1788571783.676297:0:78780:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33b.replay-single-5, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000080:00200000:0.0:1788571783.676335:0:78780:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200015bc1:0x10:0x0](ffff9ada5825e3c8) 00000080:00200000:0.0:1788571783.676435:0:78780:0:(file.c:6843:ll_layout_conf()) [0x200015bc1:0x10:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:0.0:1788571783.676441:0:78780:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200015bc1:0x10:0x0](ffff9ada5825e3c8) 00000080:00010000:0.0:1788571783.676445:0:78780:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0x10:0x0](ffff9ada5825e3c8) for lock 0xef30c340bfb3d276 00000080:00010000:0.0:1788571783.676456:0:78780:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0x10:0x0](ffff9ada5825e3c8) 00000080:00200008:0.0:1788571783.676474:0:78780:0:(file.c:1090:ll_file_open()) START file f33b.replay-single-5:[0x200015bc1:0x10:0x0](ffff9ada5825e3c8), flags 100102 00000002:00100000:0.0:1788571783.676509:0:78780:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada516f1500 x1875449613570688/t193273528361(193273528361) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571799 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:0.0:1788571783.676691:0:78780:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d276 from it ffff9ada51761a80 00010000:00010000:0.0:1788571783.676697:0:78780:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d276 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f6b2c expref: -99 pid: 78780 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.676707:0:78780:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d276 lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f6b2c expref: -99 pid: 78780 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571783.676732:0:78780:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200015bc1:0x10:0x0](ffff9ada5825e3c8), inode mode 81a4 mask 40 00000080:00200008:0.0:1788571783.676770:0:78780:0:(file.c:479:ll_file_release()) START file f33b.replay-single-5:[0x200015bc1:0x10:0x0](ffff9ada5825e3c8), flags 100002 00000002:00100000:0.0:1788571783.676942:0:78780:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9ada516f1500 x1875449613570688/t193273528361(193273528361) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571799 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:0.0:1788571783.676955:0:78780:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:0.0:1788571783.676961:0:78780:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada516f2300 x1875449613570944/t0(0) o35->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:0.0:1788571783.676989:0:78780:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f2300 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78780:1875449613570944:192.168.202.113@tcp:35:createmany.0 00000100:00100000:0.0:1788571783.677085:0:78780:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada4522e6c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788571783.688453:0:78780:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f2300 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78780:1875449613570944:192.168.202.113@tcp:35:createmany.0 00000002:00100000:0.0:1788571783.688514:0:78780:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9ada516f1500 x1875449613570688/t193273528361(193273528361) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571799 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1788571783.688620:0:78780:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571783.688644:0:78780:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571783.688719:0:78780:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d27d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78780 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.688730:0:78780:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d27d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78780 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.688742:0:78780:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d27d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78780 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571783.688766:0:78780:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f2d80 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78780:1875449613571072:192.168.202.113@tcp:101:createmany.0 00000100:00100000:0.0:1788571783.688906:0:78780:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada4522e6c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788571783.692514:0:78780:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f2d80 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78780:1875449613571072:192.168.202.113@tcp:101:createmany.0 00010000:00010000:0.0:1788571783.692711:0:78780:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d27d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6b41 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.692727:0:78780:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d27d lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6b41 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.692741:0:78780:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d27d lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6b41 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.692753:0:78780:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d27d lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6b41 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.692775:0:78780:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d27d lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6b41 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.692792:0:78780:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d27d lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6b41 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.692801:0:78780:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d27d lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6b41 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788571783.692816:0:78780:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada516f2d80 x1875449613571072/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788571799 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788571783.692834:0:78780:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6b41 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.692851:0:78780:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1788571783.692886:0:78780:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1788571783.692922:0:78780:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:0.0:1788571783.692931:0:78780:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d27d 00000080:00010000:0.0:1788571783.692937:0:78780:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571783.692941:0:78780:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d27d from it ffffab0ec145bbf8 00010000:00010000:0.0:1788571783.692945:0:78780:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6b41 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.692954:0:78780:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6b41 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571783.692990:0:78780:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000002:00010000:2.0:1788571783.697636:0:78780:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f200/0xef30c340bfb3d284 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6b5d expref: -99 pid: 78780 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.697652:0:78780:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8590023617/17 (0 0) 00000080:00200000:2.0:1788571783.697659:0:78780:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1788571783.697677:0:78780:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33b.replay-single-6, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000080:00200000:2.0:1788571783.697720:0:78780:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200015bc1:0x11:0x0](ffff9ada58197448) 00000080:00200000:2.0:1788571783.697829:0:78780:0:(file.c:6843:ll_layout_conf()) [0x200015bc1:0x11:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1788571783.697835:0:78780:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200015bc1:0x11:0x0](ffff9ada58197448) 00000080:00010000:2.0:1788571783.697840:0:78780:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0x11:0x0](ffff9ada58197448) for lock 0xef30c340bfb3d284 00000080:00010000:2.0:1788571783.697873:0:78780:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0x11:0x0](ffff9ada58197448) 00000080:00200008:2.0:1788571783.697892:0:78780:0:(file.c:1090:ll_file_open()) START file f33b.replay-single-6:[0x200015bc1:0x11:0x0](ffff9ada58197448), flags 100102 00000002:00100000:2.0:1788571783.697918:0:78780:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada603f5500 x1875449613571328/t193273528363(193273528363) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571799 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1788571783.697961:0:78780:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d284 from it ffff9ada60a45d80 00010000:00010000:2.0:1788571783.697965:0:78780:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f200/0xef30c340bfb3d284 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f6b5d expref: -99 pid: 78780 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.697975:0:78780:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f200/0xef30c340bfb3d284 lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f6b5d expref: -99 pid: 78780 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571783.697998:0:78780:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200015bc1:0x11:0x0](ffff9ada58197448), inode mode 81a4 mask 40 00000080:00200008:2.0:1788571783.698038:0:78780:0:(file.c:479:ll_file_release()) START file f33b.replay-single-6:[0x200015bc1:0x11:0x0](ffff9ada58197448), flags 100002 00000002:00100000:2.0:1788571783.698094:0:78780:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9ada603f5500 x1875449613571328/t193273528363(193273528363) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571799 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1788571783.698106:0:78780:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1788571783.698111:0:78780:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada60a36d80 x1875449613571584/t0(0) o35->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1788571783.698136:0:78780:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a36d80 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78780:1875449613571584:192.168.202.113@tcp:35:createmany.0 00000100:00100000:2.0:1788571783.698237:0:78780:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada607ffe40 going to sleep for 11 seconds 00000100:00100000:0.0:1788571783.710923:0:78780:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a36d80 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78780:1875449613571584:192.168.202.113@tcp:35:createmany.0 00000002:00100000:2.0:1788571783.712561:0:78780:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9ada603f5500 x1875449613571328/t193273528363(193273528363) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571799 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1788571783.712679:0:78780:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571783.712707:0:78780:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571783.712805:0:78780:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ca00/0xef30c340bfb3d28b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78780 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.712818:0:78780:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ca00/0xef30c340bfb3d28b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78780 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.712830:0:78780:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ca00/0xef30c340bfb3d28b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78780 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571783.712895:0:78780:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a36a00 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78780:1875449613571712:192.168.202.113@tcp:101:createmany.0 00000100:00100000:2.0:1788571783.713000:0:78780:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada607ffa80 going to sleep for 11 seconds 00000100:00100000:0.0:1788571783.716812:0:78780:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a36a00 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78780:1875449613571712:192.168.202.113@tcp:101:createmany.0 00010000:00010000:0.0:1788571783.716893:0:78780:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ca00/0xef30c340bfb3d28b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6b72 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.716908:0:78780:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ca00/0xef30c340bfb3d28b lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6b72 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.716923:0:78780:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ca00/0xef30c340bfb3d28b lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6b72 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.716935:0:78780:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ca00/0xef30c340bfb3d28b lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6b72 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.716950:0:78780:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ca00/0xef30c340bfb3d28b lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6b72 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.716959:0:78780:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ca00/0xef30c340bfb3d28b lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6b72 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.716967:0:78780:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ca00/0xef30c340bfb3d28b lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6b72 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788571783.716981:0:78780:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada60a36a00 x1875449613571712/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788571799 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788571783.716998:0:78780:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ca00/0xef30c340bfb3d28b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6b72 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571783.717014:0:78780:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1788571783.717021:0:78780:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00010000:00010000:2.0:1788571783.743325:0:78780:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f800/0xef30c340bfb3d292 lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x12:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6b8e expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.743338:0:78780:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f800/0xef30c340bfb3d292 lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6b8e expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.743353:0:78780:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f800/0xef30c340bfb3d292 lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6b8e expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.743367:0:78780:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f800/0xef30c340bfb3d292 lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6b8e expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.743384:0:78780:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f800/0xef30c340bfb3d292 lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6b8e expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.743399:0:78780:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f800/0xef30c340bfb3d292 lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6b8e expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.743408:0:78780:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f800/0xef30c340bfb3d292 lrc: 4/1,0 mode: PR/PR res: [0x200015bc1:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6b8e expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1788571783.743423:0:78780:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9ada603f7480 x1875449613571968/t193273528365(193273528365) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/664 e 0 to 0 dl 1788571799 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1788571783.743437:0:78780:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada603f7480 x1875449613571968/t193273528365(193273528365) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/664 e 0 to 0 dl 1788571799 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1788571783.743453:0:78780:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f800/0xef30c340bfb3d292 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6b8e expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1788571783.743467:0:78780:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f800/0xef30c340bfb3d292 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6b8e expref: -99 pid: 78780 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.743481:0:78780:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8590023617/18 (0 0) 00000080:00200000:2.0:1788571783.743489:0:78780:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1788571783.743505:0:78780:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33b.replay-single-7, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000080:00200000:2.0:1788571783.743541:0:78780:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200015bc1:0x12:0x0](ffff9ada581963c8) 00000080:00200000:2.0:1788571783.743641:0:78780:0:(file.c:6843:ll_layout_conf()) [0x200015bc1:0x12:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1788571783.743647:0:78780:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200015bc1:0x12:0x0](ffff9ada581963c8) 00000080:00010000:2.0:1788571783.743651:0:78780:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0x12:0x0](ffff9ada581963c8) for lock 0xef30c340bfb3d292 00000080:00010000:2.0:1788571783.743665:0:78780:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0x12:0x0](ffff9ada581963c8) 00000080:00200008:2.0:1788571783.743685:0:78780:0:(file.c:1090:ll_file_open()) START file f33b.replay-single-7:[0x200015bc1:0x12:0x0](ffff9ada581963c8), flags 100102 00000002:00100000:2.0:1788571783.743711:0:78780:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada603f7480 x1875449613571968/t193273528365(193273528365) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571799 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1788571783.743756:0:78780:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d292 from it ffff9ada60a45780 00010000:00010000:2.0:1788571783.743761:0:78780:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f800/0xef30c340bfb3d292 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f6b8e expref: -99 pid: 78780 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.743772:0:78780:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f800/0xef30c340bfb3d292 lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f6b8e expref: -99 pid: 78780 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571783.743811:0:78780:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200015bc1:0x12:0x0](ffff9ada581963c8), inode mode 81a4 mask 40 00000080:00200008:2.0:1788571783.743850:0:78780:0:(file.c:479:ll_file_release()) START file f33b.replay-single-7:[0x200015bc1:0x12:0x0](ffff9ada581963c8), flags 100002 00000002:00100000:2.0:1788571783.743940:0:78780:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9ada603f7480 x1875449613571968/t193273528365(193273528365) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571799 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1788571783.743951:0:78780:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1788571783.743958:0:78780:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada60a36a00 x1875449613572224/t0(0) o35->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1788571783.743984:0:78780:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a36a00 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78780:1875449613572224:192.168.202.113@tcp:35:createmany.0 00000100:00100000:2.0:1788571783.744107:0:78780:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada607ff900 going to sleep for 11 seconds 00000100:00100000:2.0:1788571783.749563:0:78780:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a36a00 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78780:1875449613572224:192.168.202.113@tcp:35:createmany.0 00000002:00100000:2.0:1788571783.749612:0:78780:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9ada603f7480 x1875449613571968/t193273528365(193273528365) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571799 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1788571783.749710:0:78780:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571783.749732:0:78780:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571783.749819:0:78780:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d299 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78780 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.749831:0:78780:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d299 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78780 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.749842:0:78780:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d299 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78780 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571783.749909:0:78780:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a34e00 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78780:1875449613572352:192.168.202.113@tcp:101:createmany.0 00000100:00100000:2.0:1788571783.749994:0:78780:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada607ff900 going to sleep for 11 seconds 00000100:00100000:2.0:1788571783.753623:0:78780:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a34e00 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78780:1875449613572352:192.168.202.113@tcp:101:createmany.0 00010000:00010000:2.0:1788571783.753660:0:78780:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d299 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6ba3 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.753675:0:78780:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d299 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6ba3 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.753692:0:78780:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d299 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6ba3 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.753704:0:78780:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d299 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6ba3 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.753718:0:78780:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d299 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6ba3 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.753726:0:78780:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d299 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6ba3 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.753739:0:78780:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d299 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6ba3 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1788571783.753753:0:78780:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada60a34e00 x1875449613572352/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788571799 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1788571783.753768:0:78780:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d299 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6ba3 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.753794:0:78780:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:2.0:1788571783.753803:0:78780:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:2.0:1788571783.753839:0:78780:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:2.0:1788571783.753846:0:78780:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d299 00000080:00010000:2.0:1788571783.754268:0:78780:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571783.754273:0:78780:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d299 from it ffffab0ec145bbf8 00010000:00010000:2.0:1788571783.754277:0:78780:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d299 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6ba3 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.754289:0:78780:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d299 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6ba3 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571783.754321:0:78780:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:2.0:1788571783.754350:0:78780:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571783.754369:0:78780:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571783.754379:0:78780:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d299 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6ba3 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571783.754397:0:78780:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d299 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6ba3 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571783.754404:0:78780:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571783.754409:0:78780:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571783.754412:0:78780:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d299 00000080:00010000:2.0:1788571783.754416:0:78780:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d299 from it ffffab0ec145bb38 00010000:00010000:2.0:1788571783.754419:0:78780:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d299 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6ba3 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571783.754427:0:78780:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d299 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6ba3 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571783.754434:0:78780:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:2.0:1788571783.754439:0:78780:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33b.replay-single-8, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), file ffff9ada43938f00, open_flags 8042, mode 81a4 opened 0 00000080:00200000:2.0:1788571783.754448:0:78780:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33b.replay-single-8, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000002:00010000:2.0:1788571783.754471:0:78780:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33b.replay-single-8,[0x200015bc1:0x13:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:2.0:1788571783.754484:0:78780:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d299 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b82f6ba3 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571783.754541:0:78780:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d299 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f6ba3 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571783.754549:0:78780:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1788571783.754552:0:78780:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d299 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f6ba3 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571783.754599:0:78780:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d600/0xef30c340bfb3d2a0 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78780 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.754607:0:78780:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d600/0xef30c340bfb3d2a0 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78780 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1788571783.754621:0:78780:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1788571783.754629:0:78780:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada60a34e00 x1875449613572608/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1788571783.754639:0:78780:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d600/0xef30c340bfb3d2a0 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78780 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571783.754665:0:78780:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a34e00 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78780:1875449613572608:192.168.202.113@tcp:101:createmany.0 00000100:00100000:2.0:1788571783.755499:0:78780:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada607ff900 going to sleep for 11 seconds 00000100:00100000:2.0:1788571783.770191:0:78780:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a34e00 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78780:1875449613572608:192.168.202.113@tcp:101:createmany.0 00010000:00010000:2.0:1788571783.770234:0:78780:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d600/0xef30c340bfb3d2a0 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6bbf expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.770273:0:78780:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d600/0xef30c340bfb3d2a0 lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6bbf expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.770282:0:78780:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d600/0xef30c340bfb3d2a0 lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6bbf expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.770295:0:78780:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d600/0xef30c340bfb3d2a0 lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6bbf expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.770313:0:78780:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d600/0xef30c340bfb3d2a0 lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6bbf expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.770327:0:78780:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d600/0xef30c340bfb3d2a0 lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6bbf expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.770336:0:78780:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d600/0xef30c340bfb3d2a0 lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6bbf expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.770345:0:78780:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d600/0xef30c340bfb3d2a0 lrc: 4/1,0 mode: PR/PR res: [0x200015bc1:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6bbf expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1788571783.770359:0:78780:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9ada60a34e00 x1875449613572608/t193273528367(193273528367) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/664 e 0 to 0 dl 1788571799 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1788571783.770374:0:78780:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada60a34e00 x1875449613572608/t193273528367(193273528367) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/664 e 0 to 0 dl 1788571799 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1788571783.770390:0:78780:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d600/0xef30c340bfb3d2a0 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6bbf expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1788571783.770420:0:78780:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d600/0xef30c340bfb3d2a0 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6bbf expref: -99 pid: 78780 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.770434:0:78780:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8590023617/19 (0 0) 00000080:00200000:2.0:1788571783.770442:0:78780:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1788571783.770460:0:78780:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33b.replay-single-8, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000080:00200000:2.0:1788571783.770504:0:78780:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200015bc1:0x13:0x0](ffff9ada58190908) 00000080:00200000:2.0:1788571783.770605:0:78780:0:(file.c:6843:ll_layout_conf()) [0x200015bc1:0x13:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1788571783.770611:0:78780:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200015bc1:0x13:0x0](ffff9ada58190908) 00000080:00010000:2.0:1788571783.770615:0:78780:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0x13:0x0](ffff9ada58190908) for lock 0xef30c340bfb3d2a0 00000080:00010000:2.0:1788571783.770627:0:78780:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0x13:0x0](ffff9ada58190908) 00000080:00200008:2.0:1788571783.770658:0:78780:0:(file.c:1090:ll_file_open()) START file f33b.replay-single-8:[0x200015bc1:0x13:0x0](ffff9ada58190908), flags 100102 00000002:00100000:2.0:1788571783.770685:0:78780:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada60a34e00 x1875449613572608/t193273528367(193273528367) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571799 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1788571783.770729:0:78780:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d2a0 from it ffff9ada602e3840 00010000:00010000:2.0:1788571783.770734:0:78780:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d600/0xef30c340bfb3d2a0 lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f6bbf expref: -99 pid: 78780 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.770744:0:78780:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d600/0xef30c340bfb3d2a0 lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f6bbf expref: -99 pid: 78780 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571783.770770:0:78780:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200015bc1:0x13:0x0](ffff9ada58190908), inode mode 81a4 mask 40 00000080:00200008:2.0:1788571783.770821:0:78780:0:(file.c:479:ll_file_release()) START file f33b.replay-single-8:[0x200015bc1:0x13:0x0](ffff9ada58190908), flags 100002 00000002:00100000:2.0:1788571783.770931:0:78780:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9ada60a34e00 x1875449613572608/t193273528367(193273528367) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571799 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1788571783.770943:0:78780:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1788571783.770949:0:78780:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada60a35f80 x1875449613572864/t0(0) o35->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1788571783.770971:0:78780:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35f80 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78780:1875449613572864:192.168.202.113@tcp:35:createmany.0 00000100:00100000:2.0:1788571783.771057:0:78780:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada607fff00 going to sleep for 11 seconds 00000100:00100000:2.0:1788571783.774753:0:78780:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a35f80 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78780:1875449613572864:192.168.202.113@tcp:35:createmany.0 00000002:00100000:2.0:1788571783.774975:0:78780:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9ada60a34e00 x1875449613572608/t193273528367(193273528367) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571799 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1788571783.775093:0:78780:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571783.775119:0:78780:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571783.775516:0:78780:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d2a7 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78780 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.775528:0:78780:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d2a7 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78780 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.775539:0:78780:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d2a7 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78780 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571783.775564:0:78780:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37800 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78780:1875449613572992:192.168.202.113@tcp:101:createmany.0 00000100:00100000:2.0:1788571783.775656:0:78780:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada607fff00 going to sleep for 11 seconds 00000100:00100000:2.0:1788571783.780587:0:78780:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37800 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78780:1875449613572992:192.168.202.113@tcp:101:createmany.0 00010000:00010000:2.0:1788571783.780623:0:78780:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d2a7 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6bd4 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.780638:0:78780:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d2a7 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6bd4 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.780653:0:78780:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d2a7 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6bd4 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.780666:0:78780:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d2a7 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6bd4 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.780686:0:78780:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d2a7 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6bd4 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.780695:0:78780:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d2a7 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6bd4 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.780703:0:78780:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d2a7 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6bd4 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1788571783.780716:0:78780:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada60a37800 x1875449613572992/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788571799 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1788571783.780732:0:78780:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d2a7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6bd4 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.780747:0:78780:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:2.0:1788571783.780755:0:78780:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:2.0:1788571783.780811:0:78780:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:2.0:1788571783.780819:0:78780:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d2a7 00000080:00010000:2.0:1788571783.780825:0:78780:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571783.780829:0:78780:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d2a7 from it ffffab0ec145bbf8 00010000:00010000:2.0:1788571783.780832:0:78780:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d2a7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6bd4 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.780841:0:78780:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d2a7 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6bd4 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571783.780911:0:78780:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:2.0:1788571783.780940:0:78780:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571783.780958:0:78780:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571783.780968:0:78780:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d2a7 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6bd4 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571783.780980:0:78780:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d2a7 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6bd4 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571783.780988:0:78780:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571783.780993:0:78780:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571783.780996:0:78780:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d2a7 00000080:00010000:2.0:1788571783.781000:0:78780:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d2a7 from it ffffab0ec145bb38 00010000:00010000:2.0:1788571783.781003:0:78780:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d2a7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6bd4 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571783.781012:0:78780:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d2a7 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6bd4 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571783.781021:0:78780:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:2.0:1788571783.781026:0:78780:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33b.replay-single-9, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), file ffff9ada43939e00, open_flags 8042, mode 81a4 opened 0 00000080:00200000:2.0:1788571783.781047:0:78780:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33b.replay-single-9, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000002:00010000:2.0:1788571783.781071:0:78780:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33b.replay-single-9,[0x200015bc1:0x14:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:2.0:1788571783.781094:0:78780:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d2a7 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b82f6bd4 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571783.781165:0:78780:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d2a7 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f6bd4 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571783.781174:0:78780:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1788571783.781177:0:78780:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d2a7 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f6bd4 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571783.781236:0:78780:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172da00/0xef30c340bfb3d2ae lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78780 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.781244:0:78780:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172da00/0xef30c340bfb3d2ae lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78780 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1788571783.781257:0:78780:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1788571783.781265:0:78780:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada60a37800 x1875449613573248/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1788571783.781275:0:78780:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172da00/0xef30c340bfb3d2ae lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78780 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571783.781299:0:78780:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37800 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78780:1875449613573248:192.168.202.113@tcp:101:createmany.0 00000100:00100000:2.0:1788571783.781477:0:78780:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada607fff00 going to sleep for 11 seconds 00000100:00100000:2.0:1788571783.786120:0:78780:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37800 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78780:1875449613573248:192.168.202.113@tcp:101:createmany.0 00010000:00010000:2.0:1788571783.786162:0:78780:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172da00/0xef30c340bfb3d2ae lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6bf0 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.786209:0:78780:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172da00/0xef30c340bfb3d2ae lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x14:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6bf0 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.786223:0:78780:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172da00/0xef30c340bfb3d2ae lrc: 4/0,1 mode: --/PR res: [0x200015bc1:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6bf0 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.786236:0:78780:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172da00/0xef30c340bfb3d2ae lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6bf0 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.786250:0:78780:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172da00/0xef30c340bfb3d2ae lrc: 4/0,1 mode: PR/PR res: [0x200015bc1:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6bf0 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.786267:0:78780:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172da00/0xef30c340bfb3d2ae lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6bf0 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.786276:0:78780:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172da00/0xef30c340bfb3d2ae lrc: 5/1,1 mode: PR/PR res: [0x200015bc1:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6bf0 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.786284:0:78780:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172da00/0xef30c340bfb3d2ae lrc: 4/1,0 mode: PR/PR res: [0x200015bc1:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6bf0 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1788571783.786297:0:78780:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9ada60a37800 x1875449613573248/t193273528369(193273528369) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/664 e 0 to 0 dl 1788571799 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1788571783.786311:0:78780:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada60a37800 x1875449613573248/t193273528369(193273528369) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/664 e 0 to 0 dl 1788571799 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1788571783.786326:0:78780:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172da00/0xef30c340bfb3d2ae lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6bf0 expref: -99 pid: 78780 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1788571783.786341:0:78780:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172da00/0xef30c340bfb3d2ae lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6bf0 expref: -99 pid: 78780 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.786354:0:78780:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8590023617/20 (0 0) 00000080:00200000:2.0:1788571783.786362:0:78780:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1788571783.786379:0:78780:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33b.replay-single-9, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000080:00200000:2.0:1788571783.786433:0:78780:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200015bc1:0x14:0x0](ffff9ada581921c8) 00000080:00200000:2.0:1788571783.786529:0:78780:0:(file.c:6843:ll_layout_conf()) [0x200015bc1:0x14:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1788571783.786534:0:78780:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200015bc1:0x14:0x0](ffff9ada581921c8) 00000080:00010000:2.0:1788571783.786538:0:78780:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200015bc1:0x14:0x0](ffff9ada581921c8) for lock 0xef30c340bfb3d2ae 00000080:00010000:2.0:1788571783.786549:0:78780:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200015bc1:0x14:0x0](ffff9ada581921c8) 00000080:00200008:2.0:1788571783.786570:0:78780:0:(file.c:1090:ll_file_open()) START file f33b.replay-single-9:[0x200015bc1:0x14:0x0](ffff9ada581921c8), flags 100102 00000002:00100000:2.0:1788571783.786596:0:78780:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada60a37800 x1875449613573248/t193273528369(193273528369) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571799 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1788571783.786636:0:78780:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d2ae from it ffff9ada602e3840 00010000:00010000:2.0:1788571783.786641:0:78780:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172da00/0xef30c340bfb3d2ae lrc: 3/1,0 mode: PR/PR res: [0x200015bc1:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f6bf0 expref: -99 pid: 78780 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571783.786651:0:78780:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172da00/0xef30c340bfb3d2ae lrc: 2/0,0 mode: PR/PR res: [0x200015bc1:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f6bf0 expref: -99 pid: 78780 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571783.786678:0:78780:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200015bc1:0x14:0x0](ffff9ada581921c8), inode mode 81a4 mask 40 00000080:00200008:2.0:1788571783.786714:0:78780:0:(file.c:479:ll_file_release()) START file f33b.replay-single-9:[0x200015bc1:0x14:0x0](ffff9ada581921c8), flags 100002 00000002:00100000:2.0:1788571783.786768:0:78780:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9ada60a37800 x1875449613573248/t193273528369(193273528369) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571799 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1788571783.786779:0:78780:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1788571783.786795:0:78780:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada60a37b80 x1875449613573504/t0(0) o35->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1788571783.786833:0:78780:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37b80 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78780:1875449613573504:192.168.202.113@tcp:35:createmany.0 00000100:00100000:2.0:1788571783.786943:0:78780:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada607ffcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788571783.796756:0:78780:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37b80 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:78780:1875449613573504:192.168.202.113@tcp:35:createmany.0 00000002:00100000:2.0:1788571783.796826:0:78780:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9ada60a37800 x1875449613573248/t193273528369(193273528369) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571799 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00010000:00010000:2.0:1788571785.079943:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788571786.619047:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a49f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449613573632:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571786.622642:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a4a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613573888:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571786.622700:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a49c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613574144:192.168.202.113@tcp:400:kworker.0 00010000:00010000:2.0:1788571788.151181:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571791.223167:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788571791.746293:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613574400:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571791.746316:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613574656:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571791.746330:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613574528:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571791.746341:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613574784:192.168.202.113@tcp:400:kworker.0 00000001:02000400:2.0:1788571792.238520:0:78842:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: local REPLAY BARRIER on lustre-MDT0000 00010000:00010000:0.0:1788571792.248175:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571794.295103:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788571796.866239:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a48000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449613575552:192.168.202.113@tcp:38:ptlrpcd_01_01.0 00000100:00080000:0.0:1788571796.885419:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788571796.885439:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-19) 00000100:00100000:0.0:1788571796.885446:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a48000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449613575552:192.168.202.113@tcp:38:ptlrpcd_01_01.0 00010000:00010000:2.0:1788571797.367033:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571798.393532:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571800.439744:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571801.463065:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788571801.986184:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788571801.986194:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788571801.986200:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788571801.986214:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 3714 00000100:00080000:0.0:1788571801.986229:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 5s 00000100:00080000:0.0:1788571801.986237:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788571801.986251:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788571801.986274:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada516f1f80 x1875449613575808/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788571801.986349:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449613575680:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571801.986456:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613576064:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571801.986499:0:2383:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1788571801.986505:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613575936:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571801.986529:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613576192:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571802.011020:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613575936:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571802.011038:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613576192:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571802.011051:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613576064:192.168.202.113@tcp:400:kworker.0 00010000:00010000:2.0:1788571802.487746:0:78194:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d200/0xef30c340bfb3d2b5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571802.487770:0:78194:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d200/0xef30c340bfb3d2b5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78194 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571802.488001:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00080000:2.0:1788571802.488034:0:2386:0:(client.c:1846:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != DISCONN) req@ffff9ada60a35880 x1875449613576320/t0(0) o63->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:24/4 lens 680/4320 e 0 to 0 dl 0 ref 1 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'ll_sa.0' uid:0 gid:0 projid:4294967295 00010000:00010000:0.0:1788571803.511398:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571804.536493:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571806.589242:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788571807.095617:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788571807.095625:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788571807.095629:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788571807.095639:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 3719 00000100:00080000:0.0:1788571807.095651:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 10s 00000100:00080000:0.0:1788571807.095659:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788571807.095673:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788571807.095693:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada516f3100 x1875449613576576/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788571807.095778:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613576704:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571807.095911:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613576960:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571807.095944:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1788571807.095951:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449613576448:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571807.095979:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613576832:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571807.107307:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613576832:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571807.107321:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613576704:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571807.107331:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613576960:192.168.202.113@tcp:400:kworker.0 00010000:00010000:2.0:1788571807.607552:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571809.655010:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571810.679054:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788571812.217311:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449613577088:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571812.217394:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613577472:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571812.217424:0:2383:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1788571812.217431:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613577344:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571812.217453:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613577600:192.168.202.113@tcp:400:kworker.0 00010000:00010000:0.0:1788571812.224840:0:2383:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 0/0 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1788571812.224899:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613577344:192.168.202.113@tcp:400:kworker.0 00010000:00010000:2.0:1788571812.225191:0:15205:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 0 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1788571812.225205:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571812.225206:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571812.225208:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788571812.239952:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613577600:192.168.202.113@tcp:400:kworker.0 00000100:00080000:0.0:1788571812.239974:0:2383:0:(recover.c:205:ptlrpc_request_handle_notconn()) import MGC192.168.202.113@tcp of MGS@192.168.202.113@tcp abruptly disconnected: reconnecting 00000100:00080000:0.0:1788571812.239980:0:2383:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from FULL to DISCONN 00000100:02020000:0.0:1788571812.239985:0:2383:0:(import.c:186:ptlrpc_set_import_discon()) MGC192.168.202.113@tcp: Connection to MGS (at 192.168.202.113@tcp) was lost; in progress operations using this service will fail 00000100:00080000:0.0:1788571812.239991:0:2383:0:(import.c:192:ptlrpc_set_import_discon()) import MGS@192.168.202.113@tcp for MGC192.168.202.113@tcp not replayable, auto-deactivating 00000100:00080000:0.0:1788571812.239994:0:2383:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import MGS INVALID 00000100:00100000:0.0:1788571812.240003:0:2383:0:(client.c:3591:ptlrpc_abort_inflight()) @@@ inflight req@ffff9ada60a49c00 x1875449613574912/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788571812 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788571812.240035:0:2383:0:(client.c:3591:ptlrpc_abort_inflight()) @@@ inflight req@ffff9ada516f2a00 x1875449613575680/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788571817 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788571812.240045:0:2383:0:(client.c:3591:ptlrpc_abort_inflight()) @@@ inflight req@ffff9ada516f3800 x1875449613576448/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788571823 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788571812.240062:0:2383:0:(client.c:3591:ptlrpc_abort_inflight()) @@@ inflight req@ffff9ada603f5c00 x1875449613577088/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788571828 ref 1 fl Rpc:RNQU/200/0 rc 0/-107 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:0.0:1788571812.240072:0:2383:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808001 00000100:00080000:0.0:1788571812.240076:0:2383:0:(pinger.c:193:ptlrpc_pinger_ir_down()) IR down 10000000:01000000:0.0:1788571812.240078:0:2383:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808002 00000100:00080000:0.0:1788571812.240082:0:2383:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788571812.240086:0:2383:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 3668 00000100:00080000:0.0:1788571812.240101:0:2383:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788571812.240138:0:2383:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada516f3480 x1875449613577728/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788571812.240172:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449613577088:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571812.240191:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613577472:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571812.240200:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449613575680:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571812.240507:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449613577728:192.168.202.113@tcp:250:ptlrpcd_00_00.0 00000100:00080000:2.0:1788571812.249120:0:2382:0:(import.c:1176:ptlrpc_connect_interpret()) MGC192.168.202.113@tcp: connect to target with instance 0 10000000:01000000:2.0:1788571812.249147:0:2382:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808005 00000100:02000400:2.0:1788571812.249159:0:2382:0:(import.c:1328:ptlrpc_connect_interpret()) Evicted from MGS (at 192.168.202.113@tcp) after server handle changed from 0xdb7edb52b82f6560 to 0xdb7edb52b82f6eba 00000100:00080000:2.0:1788571812.249165:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to EVICTED 00000100:00080000:2.0:1788571812.249173:0:2382:0:(import.c:1718:ptlrpc_import_recovery_state_machine()) evicted from MGS@192.168.202.113@tcp; invalidating 00000100:00080000:2.0:1788571812.250946:0:79075:0:(import.c:1642:ptlrpc_invalidate_import_thread()) thread invalidate import MGC192.168.202.113@tcp to MGS@192.168.202.113@tcp 00000100:00100000:2.0:1788571812.250964:0:79075:0:(import.c:323:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 10000000:01000000:2.0:1788571812.250969:0:79075:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808003 00010000:00010000:2.0:1788571812.250991:0:79075:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.202.113@tcp lock: ffff9ada5163fa00/0xef30c340bfb3cf3c lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x110e400000000 nid: local remote: 0xdb7edb52b82f656e expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1788571812.251005:0:79075:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.202.113@tcp lock: ffff9ada5163fa00/0xef30c340bfb3cf3c lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x110f400000000 nid: local remote: 0xdb7edb52b82f656e expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1788571812.251013:0:79075:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1788571812.251032:0:79075:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571812.251044:0:79075:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.202.113@tcp lock: ffff9ada5163fa00/0xef30c340bfb3cf3c lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x590f400000000 nid: local remote: 0xdb7edb52b82f656e expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571812.251066:0:79075:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.202.113@tcp lock: ffff9ada5163f400/0xef30c340bfb3cf4a lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0xdb7edb52b82f657c expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1788571812.251073:0:79075:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.202.113@tcp lock: ffff9ada5163f400/0xef30c340bfb3cf4a lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0xdb7edb52b82f657c expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1788571812.251079:0:79075:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1788571812.251082:0:79075:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571812.251086:0:79075:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.202.113@tcp lock: ffff9ada5163f400/0xef30c340bfb3cf4a lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0xdb7edb52b82f657c expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571812.251101:0:79075:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.202.113@tcp lock: ffff9ada5163ee00/0xef30c340bfb3cf35 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0xdb7edb52b82f6567 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1788571812.251108:0:79075:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.202.113@tcp lock: ffff9ada5163ee00/0xef30c340bfb3cf35 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0xdb7edb52b82f6567 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1788571812.251113:0:79075:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:2.0:1788571812.251117:0:79075:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571812.251120:0:79075:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.202.113@tcp lock: ffff9ada5163ee00/0xef30c340bfb3cf35 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0xdb7edb52b82f6567 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571812.251147:0:79075:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.202.113@tcp lock: ffff9ada51617e00/0xef30c340bfb3cf43 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0xdb7edb52b82f6575 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1788571812.251154:0:79075:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.202.113@tcp lock: ffff9ada51617e00/0xef30c340bfb3cf43 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0xdb7edb52b82f6575 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1788571812.251160:0:79075:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:2.0:1788571812.251164:0:79075:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571812.251168:0:79075:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.202.113@tcp lock: ffff9ada51617e00/0xef30c340bfb3cf43 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0xdb7edb52b82f6575 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1788571812.251184:0:79075:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from EVICTED to RECOVER 00000100:00080000:2.0:1788571812.251190:0:79075:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from RECOVER to FULL 10000000:01000000:2.0:1788571812.251193:0:79075:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808004 00000100:00080000:2.0:1788571812.251197:0:79075:0:(pinger.c:186:ptlrpc_pinger_ir_up()) IR up 00000100:02000000:2.0:1788571812.251201:0:79075:0:(import.c:1787:ptlrpc_import_recovery_state_machine()) MGC192.168.202.113@tcp: Connection restored to 192.168.202.113@tcp (at 192.168.202.113@tcp) 00010000:00080000:2.0:1788571812.252515:0:2382:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff9ada603f7b80 x1875449613577216/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/416 e 0 to 0 dl 1788571837 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788571812.252736:0:2382:0:(import.c:1176:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to target with instance 46 00000100:00080000:2.0:1788571812.252764:0:2382:0:(import.c:1382:ptlrpc_connect_interpret()) @@@ lustre-MDT0000-mdc-ffff9ada58b0a000: evicting (reconnect/recover flags not set: 4) req@ffff9ada603f7b80 x1875449613577216/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/416 e 0 to 0 dl 1788571837 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788571812.252773:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to EVICTED 00000100:02020000:2.0:1788571812.252780:0:2382:0:(import.c:1707:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff9ada58b0a000: This client was evicted by lustre-MDT0000; in progress operations using this service will fail. 00000100:00080000:2.0:1788571812.274125:0:2382:0:(import.c:1718:ptlrpc_import_recovery_state_machine()) evicted from lustre-MDT0000_UUID@192.168.202.113@tcp; invalidating 00000100:00100000:0.0:1788571812.277493:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449613577216:192.168.202.113@tcp:38:kworker.0 00010000:00010000:2.0:1788571812.733336:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1788571814.130005:0:79116:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x2:0x0] index 64768 count 7 00010000:00010000:2.0:1788571814.775173:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571815.799915:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571817.846992:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571818.872042:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571820.920122:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571821.943134:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788571822.478821:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613580800:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571822.478903:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449613580416:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571822.478926:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613580544:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571822.479017:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613580672:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571822.479030:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613580928:192.168.202.113@tcp:400:kworker.0 00000080:00200080:0.0:1788571823.098523:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.098559:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00800000:00000004:0.0:1788571823.098571:0:79219:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: age 3738, max_age 3739 00000100:00100000:0.0:1788571823.099467:0:79219:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f2680 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:79219:1875449613581184:192.168.202.113@tcp:41:lfs.0 00000100:00100000:0.0:1788571823.099582:0:79219:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada44fdfc00 going to sleep for 11 seconds 00000100:00100000:0.0:1788571823.101835:0:79219:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f2680 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:79219:1875449613581184:192.168.202.113@tcp:41:lfs.0 00800000:00000004:0.0:1788571823.101899:0:79219:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: update ffff9ada44819e18 cache blocks 317416/353529 objects 1023739/1024000 00000080:00200080:0.0:1788571823.102014:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102033:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102041:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102044:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102048:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102051:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102055:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102058:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102061:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102064:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102067:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102071:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102074:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102077:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102081:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102083:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102086:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102089:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102092:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102094:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102097:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102100:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102103:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102105:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102108:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102110:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102113:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102116:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102119:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102121:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102125:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102127:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102130:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102133:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102136:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102138:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102142:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102145:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102148:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102152:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102156:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102160:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102232:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102239:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102244:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102248:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102252:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102255:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102258:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102261:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102265:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102268:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102271:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102275:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102278:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102282:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102286:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102289:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102293:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102296:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102300:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102303:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102307:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102311:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102315:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102319:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102322:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102326:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102329:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102333:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102337:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102341:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102345:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102348:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102352:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102356:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102359:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102362:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102366:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102369:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102373:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102377:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102380:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102384:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102388:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102392:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102397:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102400:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102404:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102408:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102412:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102415:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102419:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102422:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102426:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102429:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102433:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102436:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102440:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102444:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102448:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102451:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102455:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102458:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102462:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102466:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102470:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102473:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102476:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102480:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102483:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102487:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102491:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102494:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102498:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102502:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102506:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102510:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102514:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102518:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102526:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102531:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102535:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102538:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102542:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00800000:00000080:0.0:1788571823.102556:0:79219:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.102562:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.102569:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00020000:00000004:0.0:1788571823.102579:0:79219:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: age 3738, max_age 3739 00000100:00100000:0.0:1788571823.103134:0:79219:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f2680 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:79219:1875449613581312:192.168.202.113@tcp:13:lfs.0 00000100:00100000:0.0:1788571823.103358:0:79219:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada44fdfc00 going to sleep for 11 seconds 00000100:00100000:0.0:1788571823.108431:0:79219:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f2680 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:79219:1875449613581312:192.168.202.113@tcp:13:lfs.0 00020000:00000004:0.0:1788571823.108466:0:79219:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: update ffff9ada4354d198 cache blocks 901357/958279 objects 261729/262144 00000080:00200080:0.0:1788571823.108571:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.108606:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00020000:00000004:0.0:1788571823.108619:0:79219:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: age 3738, max_age 3739 00000100:00100000:0.0:1788571823.108673:0:79219:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f2680 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:79219:1875449613581440:192.168.202.113@tcp:13:lfs.0 00000100:00100000:0.0:1788571823.108770:0:79219:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada44fdfc00 going to sleep for 11 seconds 00000100:00100000:0.0:1788571823.113095:0:79219:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f2680 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:79219:1875449613581440:192.168.202.113@tcp:13:lfs.0 00020000:00000004:0.0:1788571823.113138:0:79219:0:(obd_class.h:1116:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: update ffff9ada43549e18 cache blocks 901363/958279 objects 261734/262144 00000080:00200080:0.0:1788571823.113297:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113617:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113630:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113634:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113639:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113643:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113647:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113650:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113654:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113657:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113661:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113665:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113669:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113672:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113676:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113679:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113683:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113687:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113691:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113695:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113699:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113703:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113707:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113710:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113714:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113718:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113722:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113726:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113730:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113734:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113738:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113741:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113745:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113749:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113753:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113757:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113761:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113764:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113768:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113771:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113775:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113779:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113783:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113786:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113790:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113794:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113798:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113801:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113805:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113808:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113813:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113816:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113820:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113823:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113828:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113831:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113835:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113838:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113842:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113845:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113849:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113899:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113907:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113911:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113916:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113919:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113924:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113927:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113931:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113936:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113939:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113943:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113947:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113950:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113954:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113958:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113961:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113965:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113969:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113973:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113977:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113980:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113983:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113987:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113991:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.113994:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.113998:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.114001:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.114005:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.114009:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.114013:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.114016:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.114020:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.114023:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.114027:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.114030:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.114034:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.114037:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.114041:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.114045:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.114049:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.114052:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.114057:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.114060:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.114064:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.114068:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.114071:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.114074:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.114078:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.114081:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.114084:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.114088:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.114091:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.114094:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.114098:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.114102:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.114105:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.114108:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.114112:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.114116:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.114120:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.114123:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.114127:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.114130:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200080:0.0:1788571823.114134:0:79219:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe4abfed50 00020000:00000080:0.0:1788571823.114137:0:79219:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a3400 uarg=0000000000000000 00000080:00200008:0.0:1788571823.114165:0:79219:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00010000:00010000:2.0:1788571823.991393:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571825.015144:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571827.063142:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788571827.584181:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a49f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613582080:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571827.584205:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a49880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613581952:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571827.587114:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a49c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613581824:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571827.587143:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a48e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613581696:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571827.599093:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a48380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449613581568:192.168.202.113@tcp:400:kworker.0 00010000:00010000:2.0:1788571828.087196:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571830.139328:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571831.160269:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:2.0:1788571832.381833:0:79356:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571832.381909:0:79356:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571832.381937:0:79356:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e200/0xef30c340bfb3d2d8 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6fcb expref: -99 pid: 79167 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:2.0:1788571832.381958:0:79356:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e200/0xef30c340bfb3d2d8 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6fcb expref: -99 pid: 79167 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000080:00200000:2.0:1788571832.381969:0:79356:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571832.382335:0:79356:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571832.382340:0:79356:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d2d8 00000080:00010000:2.0:1788571832.382350:0:79356:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d2d8 from it ffffab0ec1cbfbf8 00010000:00010000:2.0:1788571832.382355:0:79356:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e200/0xef30c340bfb3d2d8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6fcb expref: -99 pid: 79167 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:2.0:1788571832.382366:0:79356:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e200/0xef30c340bfb3d2d8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6fcb expref: -99 pid: 79167 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00000080:00200000:2.0:1788571832.382389:0:79356:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:2.0:1788571832.382411:0:79356:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571832.382419:0:79356:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571832.382426:0:79356:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e200/0xef30c340bfb3d2d8 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6fcb expref: -99 pid: 79167 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00010000:00010000:2.0:1788571832.382434:0:79356:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e200/0xef30c340bfb3d2d8 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6fcb expref: -99 pid: 79167 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00000080:00200000:2.0:1788571832.382441:0:79356:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571832.382445:0:79356:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571832.382447:0:79356:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d2d8 00000080:00010000:2.0:1788571832.382452:0:79356:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d2d8 from it ffffab0ec1cbfb38 00010000:00010000:2.0:1788571832.382455:0:79356:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e200/0xef30c340bfb3d2d8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6fcb expref: -99 pid: 79167 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00010000:00010000:2.0:1788571832.382463:0:79356:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e200/0xef30c340bfb3d2d8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6fcb expref: -99 pid: 79167 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00000080:00200000:2.0:1788571832.382472:0:79356:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:2.0:1788571832.382478:0:79356:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33b.replay-single--0, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), file ffff9ada616b5c80, open_flags 8042, mode 81a4 opened 0 00000080:00200000:2.0:1788571832.383205:0:79356:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33b.replay-single--0, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000100:00100000:2.0:1788571832.383311:0:79356:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37100 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:79356:1875449613582208:192.168.202.113@tcp:700:createmany.0 00000100:00100000:2.0:1788571832.383462:0:79356:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6059b840 going to sleep for 11 seconds 00000100:00100000:2.0:1788571832.404784:0:79356:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37100 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:79356:1875449613582208:192.168.202.113@tcp:700:createmany.0 00000002:00010000:2.0:1788571832.404846:0:79356:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33b.replay-single--0,[0x200016b60:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:2.0:1788571832.404921:0:79356:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e200/0xef30c340bfb3d2d8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b82f6fcb expref: -99 pid: 79167 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:2.0:1788571832.405013:0:79356:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e200/0xef30c340bfb3d2d8 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f6fcb expref: -99 pid: 79167 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:2.0:1788571832.405021:0:79356:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1788571832.405025:0:79356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e200/0xef30c340bfb3d2d8 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f6fcb expref: -99 pid: 79167 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:2.0:1788571832.405092:0:79356:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d200/0xef30c340bfb3d2df lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 79356 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.405100:0:79356:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d200/0xef30c340bfb3d2df lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79356 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1788571832.405112:0:79356:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1788571832.405122:0:79356:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada60a37100 x1875449613582464/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1788571832.405132:0:79356:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d200/0xef30c340bfb3d2df lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79356 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571832.405156:0:79356:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37100 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:79356:1875449613582464:192.168.202.113@tcp:101:createmany.0 00000100:00100000:2.0:1788571832.405331:0:79356:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6059b840 going to sleep for 11 seconds 00000100:00100000:2.0:1788571832.416548:0:79356:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37100 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:79356:1875449613582464:192.168.202.113@tcp:101:createmany.0 00010000:00010000:2.0:1788571832.416597:0:79356:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d200/0xef30c340bfb3d2df lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6fe7 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.416638:0:79356:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d200/0xef30c340bfb3d2df lrc: 4/0,1 mode: --/PR res: [0x200016b60:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6fe7 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.416648:0:79356:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d200/0xef30c340bfb3d2df lrc: 4/0,1 mode: --/PR res: [0x200016b60:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6fe7 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.416662:0:79356:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d200/0xef30c340bfb3d2df lrc: 4/0,1 mode: PR/PR res: [0x200016b60:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6fe7 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.416675:0:79356:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d200/0xef30c340bfb3d2df lrc: 4/0,1 mode: PR/PR res: [0x200016b60:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6fe7 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.416691:0:79356:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d200/0xef30c340bfb3d2df lrc: 5/1,1 mode: PR/PR res: [0x200016b60:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6fe7 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.416698:0:79356:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d200/0xef30c340bfb3d2df lrc: 5/1,1 mode: PR/PR res: [0x200016b60:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6fe7 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.416706:0:79356:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d200/0xef30c340bfb3d2df lrc: 4/1,0 mode: PR/PR res: [0x200016b60:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6fe7 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1788571832.416720:0:79356:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9ada60a37100 x1875449613582464/t197568495617(197568495617) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/664 e 0 to 0 dl 1788571848 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1788571832.416744:0:79356:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada60a37100 x1875449613582464/t197568495617(197568495617) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/664 e 0 to 0 dl 1788571848 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1788571832.416761:0:79356:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d200/0xef30c340bfb3d2df lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6fe7 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1788571832.416777:0:79356:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d200/0xef30c340bfb3d2df lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6fe7 expref: -99 pid: 79356 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.416792:0:79356:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8590027616/1 (0 0) 00000080:00200000:2.0:1788571832.416808:0:79356:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1788571832.416827:0:79356:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33b.replay-single--0, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000080:00200000:2.0:1788571832.416930:0:79356:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200016b60:0x1:0x0](ffff9ada58193248) 00000080:00200000:2.0:1788571832.417265:0:79356:0:(file.c:6843:ll_layout_conf()) [0x200016b60:0x1:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1788571832.417271:0:79356:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200016b60:0x1:0x0](ffff9ada58193248) 00000080:00010000:2.0:1788571832.417275:0:79356:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200016b60:0x1:0x0](ffff9ada58193248) for lock 0xef30c340bfb3d2df 00000080:00010000:2.0:1788571832.417287:0:79356:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200016b60:0x1:0x0](ffff9ada58193248) 00000080:00200008:2.0:1788571832.417305:0:79356:0:(file.c:1090:ll_file_open()) START file f33b.replay-single--0:[0x200016b60:0x1:0x0](ffff9ada58193248), flags 100102 00000002:00100000:2.0:1788571832.417332:0:79356:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada60a37100 x1875449613582464/t197568495617(197568495617) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571848 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1788571832.417378:0:79356:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d2df from it ffff9ada602e3600 00010000:00010000:2.0:1788571832.417383:0:79356:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d200/0xef30c340bfb3d2df lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f6fe7 expref: -99 pid: 79356 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.417394:0:79356:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d200/0xef30c340bfb3d2df lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f6fe7 expref: -99 pid: 79356 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571832.417421:0:79356:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200016b60:0x1:0x0](ffff9ada58193248), inode mode 81a4 mask 40 00000080:00200008:2.0:1788571832.417459:0:79356:0:(file.c:479:ll_file_release()) START file f33b.replay-single--0:[0x200016b60:0x1:0x0](ffff9ada58193248), flags 100002 00000002:00100000:2.0:1788571832.417516:0:79356:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9ada60a37100 x1875449613582464/t197568495617(197568495617) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571848 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1788571832.417535:0:79356:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1788571832.417541:0:79356:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada60a35500 x1875449613582720/t0(0) o35->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1788571832.417569:0:79356:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35500 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:79356:1875449613582720:192.168.202.113@tcp:35:createmany.0 00000100:00100000:2.0:1788571832.417675:0:79356:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6059be40 going to sleep for 11 seconds 00000100:00100000:2.0:1788571832.429601:0:79356:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a35500 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:79356:1875449613582720:192.168.202.113@tcp:35:createmany.0 00000002:00100000:2.0:1788571832.429655:0:79356:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9ada60a37100 x1875449613582464/t197568495617(197568495617) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571848 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1788571832.429776:0:79356:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571832.429801:0:79356:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00000100:00100000:0.0:1788571832.439709:0:79356:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a34700 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:79356:1875449613582848:192.168.202.113@tcp:101:createmany.0 00010000:00010000:0.0:1788571832.439761:0:79356:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759a00/0xef30c340bfb3d2e6 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6ffc expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.439776:0:79356:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759a00/0xef30c340bfb3d2e6 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6ffc expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.439792:0:79356:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759a00/0xef30c340bfb3d2e6 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6ffc expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.439805:0:79356:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759a00/0xef30c340bfb3d2e6 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6ffc expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.439820:0:79356:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759a00/0xef30c340bfb3d2e6 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6ffc expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.439829:0:79356:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759a00/0xef30c340bfb3d2e6 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6ffc expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.439843:0:79356:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759a00/0xef30c340bfb3d2e6 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6ffc expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788571832.439907:0:79356:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada60a34700 x1875449613582848/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788571848 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788571832.439927:0:79356:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759a00/0xef30c340bfb3d2e6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6ffc expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.439943:0:79356:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1788571832.439951:0:79356:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1788571832.440003:0:79356:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:0.0:1788571832.440012:0:79356:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d2e6 00000080:00010000:0.0:1788571832.440020:0:79356:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571832.440024:0:79356:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d2e6 from it ffffab0ec1cbfbf8 00010000:00010000:0.0:1788571832.440027:0:79356:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759a00/0xef30c340bfb3d2e6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6ffc expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.440045:0:79356:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759a00/0xef30c340bfb3d2e6 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6ffc expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571832.440079:0:79356:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:0.0:1788571832.440112:0:79356:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571832.440136:0:79356:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571832.440147:0:79356:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759a00/0xef30c340bfb3d2e6 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6ffc expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571832.440164:0:79356:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759a00/0xef30c340bfb3d2e6 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6ffc expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571832.440172:0:79356:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788571832.440177:0:79356:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571832.440180:0:79356:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d2e6 00000080:00010000:0.0:1788571832.440184:0:79356:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d2e6 from it ffffab0ec1cbfb38 00010000:00010000:0.0:1788571832.440187:0:79356:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759a00/0xef30c340bfb3d2e6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f6ffc expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571832.440195:0:79356:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759a00/0xef30c340bfb3d2e6 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f6ffc expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788571832.440203:0:79356:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:0.0:1788571832.440209:0:79356:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33b.replay-single--1, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), file ffff9ada616b4d80, open_flags 8042, mode 81a4 opened 0 00000080:00200000:0.0:1788571832.440220:0:79356:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33b.replay-single--1, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000002:00010000:0.0:1788571832.440249:0:79356:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33b.replay-single--1,[0x200016b60:0x2:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:0.0:1788571832.440262:0:79356:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759a00/0xef30c340bfb3d2e6 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b82f6ffc expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571832.440322:0:79356:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759a00/0xef30c340bfb3d2e6 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f6ffc expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571832.440331:0:79356:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1788571832.440334:0:79356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759a00/0xef30c340bfb3d2e6 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f6ffc expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571832.440389:0:79356:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d2ed lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 79356 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.440399:0:79356:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d2ed lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79356 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:0.0:1788571832.440415:0:79356:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:0.0:1788571832.440424:0:79356:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada516f2680 x1875449613583104/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:0.0:1788571832.440434:0:79356:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d2ed lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79356 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571832.440461:0:79356:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f2680 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:79356:1875449613583104:192.168.202.113@tcp:101:createmany.0 00000100:00100000:0.0:1788571832.440645:0:79356:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada44fdff00 going to sleep for 11 seconds 00000100:00100000:0.0:1788571832.451788:0:79356:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f2680 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:79356:1875449613583104:192.168.202.113@tcp:101:createmany.0 00010000:00010000:0.0:1788571832.451835:0:79356:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d2ed lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7018 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.451922:0:79356:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d2ed lrc: 4/0,1 mode: --/PR res: [0x200016b60:0x2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7018 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.451932:0:79356:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d2ed lrc: 4/0,1 mode: --/PR res: [0x200016b60:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7018 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.451945:0:79356:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d2ed lrc: 4/0,1 mode: PR/PR res: [0x200016b60:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7018 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.451964:0:79356:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d2ed lrc: 4/0,1 mode: PR/PR res: [0x200016b60:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7018 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.451991:0:79356:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d2ed lrc: 5/1,1 mode: PR/PR res: [0x200016b60:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7018 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.452002:0:79356:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d2ed lrc: 5/1,1 mode: PR/PR res: [0x200016b60:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7018 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.452011:0:79356:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d2ed lrc: 4/1,0 mode: PR/PR res: [0x200016b60:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7018 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788571832.452025:0:79356:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9ada516f2680 x1875449613583104/t197568495619(197568495619) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/664 e 0 to 0 dl 1788571848 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:0.0:1788571832.452228:0:79356:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada516f2680 x1875449613583104/t197568495619(197568495619) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/664 e 0 to 0 dl 1788571848 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788571832.452246:0:79356:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d2ed lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7018 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:0.0:1788571832.452261:0:79356:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d2ed lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7018 expref: -99 pid: 79356 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.452275:0:79356:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8590027616/2 (0 0) 00000080:00200000:0.0:1788571832.452283:0:79356:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1788571832.452301:0:79356:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33b.replay-single--1, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000080:00200000:0.0:1788571832.452362:0:79356:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200016b60:0x2:0x0](ffff9ada58217448) 00000080:00200000:0.0:1788571832.452461:0:79356:0:(file.c:6843:ll_layout_conf()) [0x200016b60:0x2:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:0.0:1788571832.452467:0:79356:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200016b60:0x2:0x0](ffff9ada58217448) 00000080:00010000:0.0:1788571832.452470:0:79356:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200016b60:0x2:0x0](ffff9ada58217448) for lock 0xef30c340bfb3d2ed 00000080:00010000:0.0:1788571832.452484:0:79356:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200016b60:0x2:0x0](ffff9ada58217448) 00000080:00200008:0.0:1788571832.452507:0:79356:0:(file.c:1090:ll_file_open()) START file f33b.replay-single--1:[0x200016b60:0x2:0x0](ffff9ada58217448), flags 100102 00000002:00100000:0.0:1788571832.452535:0:79356:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada516f2680 x1875449613583104/t197568495619(197568495619) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571848 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:0.0:1788571832.452698:0:79356:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d2ed from it ffff9ada51761cc0 00010000:00010000:0.0:1788571832.452704:0:79356:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d2ed lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f7018 expref: -99 pid: 79356 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.452715:0:79356:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d2ed lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f7018 expref: -99 pid: 79356 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571832.452752:0:79356:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200016b60:0x2:0x0](ffff9ada58217448), inode mode 81a4 mask 40 00000080:00200008:0.0:1788571832.452793:0:79356:0:(file.c:479:ll_file_release()) START file f33b.replay-single--1:[0x200016b60:0x2:0x0](ffff9ada58217448), flags 100002 00000002:00100000:0.0:1788571832.452848:0:79356:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9ada516f2680 x1875449613583104/t197568495619(197568495619) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571848 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:0.0:1788571832.453332:0:79356:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:0.0:1788571832.453340:0:79356:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada516f3b80 x1875449613583360/t0(0) o35->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:0.0:1788571832.453382:0:79356:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f3b80 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:79356:1875449613583360:192.168.202.113@tcp:35:createmany.0 00000100:00100000:0.0:1788571832.453504:0:79356:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada44fdfcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788571832.459955:0:79356:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f3b80 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:79356:1875449613583360:192.168.202.113@tcp:35:createmany.0 00000002:00100000:0.0:1788571832.460018:0:79356:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9ada516f2680 x1875449613583104/t197568495619(197568495619) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571848 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1788571832.460133:0:79356:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571832.460158:0:79356:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571832.460233:0:79356:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d2f4 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 79356 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.460244:0:79356:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d2f4 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79356 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.460255:0:79356:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d2f4 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79356 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571832.460277:0:79356:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0000 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:79356:1875449613583488:192.168.202.113@tcp:101:createmany.0 00000100:00100000:0.0:1788571832.460366:0:79356:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada44fdfcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788571832.463251:0:79356:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0000 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:79356:1875449613583488:192.168.202.113@tcp:101:createmany.0 00010000:00010000:0.0:1788571832.463291:0:79356:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d2f4 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f702d expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.463305:0:79356:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d2f4 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f702d expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.463321:0:79356:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d2f4 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f702d expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.463335:0:79356:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d2f4 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f702d expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.463373:0:79356:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d2f4 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f702d expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.463382:0:79356:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d2f4 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f702d expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.463391:0:79356:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d2f4 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f702d expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788571832.463408:0:79356:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada516f0000 x1875449613583488/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788571848 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788571832.463426:0:79356:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d2f4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f702d expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.463443:0:79356:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1788571832.463453:0:79356:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1788571832.463495:0:79356:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:0.0:1788571832.463504:0:79356:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d2f4 00000080:00010000:0.0:1788571832.463510:0:79356:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571832.463514:0:79356:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d2f4 from it ffffab0ec1cbfbf8 00010000:00010000:0.0:1788571832.463518:0:79356:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d2f4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f702d expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.463527:0:79356:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d2f4 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f702d expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571832.463563:0:79356:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:0.0:1788571832.463597:0:79356:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571832.463617:0:79356:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571832.463627:0:79356:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d2f4 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f702d expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571832.463639:0:79356:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d2f4 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f702d expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571832.463646:0:79356:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788571832.463651:0:79356:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571832.463654:0:79356:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d2f4 00000080:00010000:0.0:1788571832.463657:0:79356:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d2f4 from it ffffab0ec1cbfb38 00010000:00010000:0.0:1788571832.463661:0:79356:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d2f4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f702d expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571832.463668:0:79356:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d2f4 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f702d expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788571832.463678:0:79356:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:0.0:1788571832.463683:0:79356:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33b.replay-single--2, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), file ffff9ada600b2c00, open_flags 8042, mode 81a4 opened 0 00000080:00200000:0.0:1788571832.463692:0:79356:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33b.replay-single--2, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000002:00010000:0.0:1788571832.463716:0:79356:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33b.replay-single--2,[0x200016b60:0x3:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:0.0:1788571832.463738:0:79356:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d2f4 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b82f702d expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571832.463803:0:79356:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d2f4 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f702d expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571832.463811:0:79356:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1788571832.463814:0:79356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d2f4 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f702d expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571832.463891:0:79356:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d2fb lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 79356 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.463899:0:79356:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d2fb lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79356 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:0.0:1788571832.463911:0:79356:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:0.0:1788571832.463919:0:79356:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada516f0000 x1875449613583744/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:0.0:1788571832.463929:0:79356:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d2fb lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79356 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571832.463957:0:79356:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0000 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:79356:1875449613583744:192.168.202.113@tcp:101:createmany.0 00000100:00100000:0.0:1788571832.465848:0:79356:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada44fdfcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788571832.483205:0:79356:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0000 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:79356:1875449613583744:192.168.202.113@tcp:101:createmany.0 00010000:00010000:0.0:1788571832.483244:0:79356:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d2fb lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7049 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.483287:0:79356:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d2fb lrc: 4/0,1 mode: --/PR res: [0x200016b60:0x3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7049 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.483301:0:79356:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d2fb lrc: 4/0,1 mode: --/PR res: [0x200016b60:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7049 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.483314:0:79356:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d2fb lrc: 4/0,1 mode: PR/PR res: [0x200016b60:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7049 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.483326:0:79356:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d2fb lrc: 4/0,1 mode: PR/PR res: [0x200016b60:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7049 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.483342:0:79356:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d2fb lrc: 5/1,1 mode: PR/PR res: [0x200016b60:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7049 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.484545:0:79356:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d2fb lrc: 5/1,1 mode: PR/PR res: [0x200016b60:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7049 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.484558:0:79356:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d2fb lrc: 4/1,0 mode: PR/PR res: [0x200016b60:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7049 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788571832.484589:0:79356:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9ada516f0000 x1875449613583744/t197568495621(197568495621) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/664 e 0 to 0 dl 1788571848 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:0.0:1788571832.484604:0:79356:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada516f0000 x1875449613583744/t197568495621(197568495621) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/664 e 0 to 0 dl 1788571848 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788571832.484622:0:79356:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d2fb lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7049 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:0.0:1788571832.484641:0:79356:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d2fb lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7049 expref: -99 pid: 79356 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.484658:0:79356:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8590027616/3 (0 0) 00000080:00200000:0.0:1788571832.484665:0:79356:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1788571832.484682:0:79356:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33b.replay-single--2, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000080:00200000:0.0:1788571832.484716:0:79356:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200016b60:0x3:0x0](ffff9ada582163c8) 00000080:00200000:0.0:1788571832.484844:0:79356:0:(file.c:6843:ll_layout_conf()) [0x200016b60:0x3:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:0.0:1788571832.484850:0:79356:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200016b60:0x3:0x0](ffff9ada582163c8) 00000080:00010000:0.0:1788571832.484879:0:79356:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200016b60:0x3:0x0](ffff9ada582163c8) for lock 0xef30c340bfb3d2fb 00000080:00010000:0.0:1788571832.484896:0:79356:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200016b60:0x3:0x0](ffff9ada582163c8) 00000080:00200008:0.0:1788571832.484916:0:79356:0:(file.c:1090:ll_file_open()) START file f33b.replay-single--2:[0x200016b60:0x3:0x0](ffff9ada582163c8), flags 100102 00000002:00100000:0.0:1788571832.484945:0:79356:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada516f0000 x1875449613583744/t197568495621(197568495621) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571848 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:0.0:1788571832.485004:0:79356:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d2fb from it ffff9ada51761cc0 00010000:00010000:0.0:1788571832.485009:0:79356:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d2fb lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f7049 expref: -99 pid: 79356 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.485022:0:79356:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616e00/0xef30c340bfb3d2fb lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f7049 expref: -99 pid: 79356 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571832.486297:0:79356:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200016b60:0x3:0x0](ffff9ada582163c8), inode mode 81a4 mask 40 00000080:00200008:0.0:1788571832.486365:0:79356:0:(file.c:479:ll_file_release()) START file f33b.replay-single--2:[0x200016b60:0x3:0x0](ffff9ada582163c8), flags 100002 00000002:00100000:0.0:1788571832.486456:0:79356:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9ada516f0000 x1875449613583744/t197568495621(197568495621) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571848 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:0.0:1788571832.486471:0:79356:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:0.0:1788571832.486477:0:79356:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada516f0a80 x1875449613584000/t0(0) o35->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:0.0:1788571832.486513:0:79356:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0a80 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:79356:1875449613584000:192.168.202.113@tcp:35:createmany.0 00000100:00100000:0.0:1788571832.486635:0:79356:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada44fdf600 going to sleep for 11 seconds 00000100:00100000:0.0:1788571832.490805:0:79356:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0a80 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:79356:1875449613584000:192.168.202.113@tcp:35:createmany.0 00000002:00100000:0.0:1788571832.490878:0:79356:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9ada516f0000 x1875449613583744/t197568495621(197568495621) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571848 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1788571832.491098:0:79356:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571832.491127:0:79356:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571832.491206:0:79356:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d302 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 79356 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.491218:0:79356:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d302 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79356 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.491230:0:79356:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d302 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79356 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571832.491256:0:79356:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f2a00 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:79356:1875449613584128:192.168.202.113@tcp:101:createmany.0 00000100:00100000:0.0:1788571832.491341:0:79356:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada44fdf600 going to sleep for 11 seconds 00000100:00100000:0.0:1788571832.495706:0:79356:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f2a00 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:79356:1875449613584128:192.168.202.113@tcp:101:createmany.0 00010000:00010000:0.0:1788571832.495756:0:79356:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d302 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f705e expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.495770:0:79356:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d302 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f705e expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.495790:0:79356:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d302 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f705e expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.495813:0:79356:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d302 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f705e expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.495830:0:79356:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d302 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f705e expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.495838:0:79356:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d302 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f705e expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.495846:0:79356:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d302 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f705e expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788571832.495902:0:79356:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada516f2a00 x1875449613584128/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788571848 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788571832.495919:0:79356:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d302 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f705e expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.495935:0:79356:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1788571832.495944:0:79356:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:2.0:1788571832.501260:0:79356:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:2.0:1788571832.501291:0:79356:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d302 00000080:00010000:2.0:1788571832.501303:0:79356:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571832.501307:0:79356:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d302 from it ffffab0ec1cbfbf8 00010000:00010000:2.0:1788571832.501316:0:79356:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d302 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f705e expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.501334:0:79356:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d302 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f705e expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571832.501385:0:79356:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:2.0:1788571832.501430:0:79356:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571832.501456:0:79356:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571832.501476:0:79356:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d302 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f705e expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571832.501492:0:79356:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d302 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f705e expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571832.501501:0:79356:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571832.501509:0:79356:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571832.501512:0:79356:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d302 00000080:00010000:2.0:1788571832.501516:0:79356:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d302 from it ffffab0ec1cbfb38 00010000:00010000:2.0:1788571832.501519:0:79356:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d302 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f705e expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571832.501527:0:79356:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d302 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f705e expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571832.501537:0:79356:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:2.0:1788571832.501544:0:79356:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33b.replay-single--3, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), file ffff9ada600b2d80, open_flags 8042, mode 81a4 opened 0 00000080:00200000:2.0:1788571832.501554:0:79356:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33b.replay-single--3, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000002:00010000:2.0:1788571832.501593:0:79356:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33b.replay-single--3,[0x200016b60:0x4:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:2.0:1788571832.501608:0:79356:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d302 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b82f705e expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571832.501677:0:79356:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d302 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f705e expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571832.501685:0:79356:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1788571832.501688:0:79356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d302 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f705e expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571832.501757:0:79356:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d309 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 79356 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.501766:0:79356:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d309 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79356 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1788571832.501781:0:79356:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1788571832.502223:0:79356:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada60a35180 x1875449613584384/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1788571832.502237:0:79356:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d309 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79356 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571832.502267:0:79356:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35180 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:79356:1875449613584384:192.168.202.113@tcp:101:createmany.0 00000100:00100000:2.0:1788571832.502470:0:79356:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6059be40 going to sleep for 11 seconds 00010000:00010000:2.0:1788571832.529720:0:79356:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759800/0xef30c340bfb3d310 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f708f expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571832.529740:0:79356:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1788571832.529744:0:79356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759800/0xef30c340bfb3d310 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f708f expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571832.529794:0:79356:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d317 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 79356 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.529802:0:79356:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d317 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79356 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1788571832.529815:0:79356:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1788571832.529826:0:79356:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada60a4a300 x1875449613585024/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1788571832.529836:0:79356:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d317 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79356 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571832.529965:0:79356:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a4a300 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:79356:1875449613585024:192.168.202.113@tcp:101:createmany.0 00000100:00100000:2.0:1788571832.530152:0:79356:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6059b180 going to sleep for 11 seconds 00000080:00200000:2.0:1788571832.547745:0:79356:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200016b60:0x5:0x0](ffff9ada581980c8), inode mode 81a4 mask 40 00000080:00200008:2.0:1788571832.547819:0:79356:0:(file.c:479:ll_file_release()) START file f33b.replay-single--4:[0x200016b60:0x5:0x0](ffff9ada581980c8), flags 100002 00000002:00100000:2.0:1788571832.549146:0:79356:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9ada60a4a300 x1875449613585024/t197568495625(197568495625) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571848 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1788571832.549168:0:79356:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1788571832.549175:0:79356:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada60a35c00 x1875449613585280/t0(0) o35->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1788571832.549207:0:79356:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35c00 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:79356:1875449613585280:192.168.202.113@tcp:35:createmany.0 00000100:00100000:2.0:1788571832.549386:0:79356:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6059b0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788571832.562775:0:79356:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a35c00 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:79356:1875449613585280:192.168.202.113@tcp:35:createmany.0 00000002:00100000:0.0:1788571832.562831:0:79356:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9ada60a4a300 x1875449613585024/t197568495625(197568495625) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571848 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1788571832.563011:0:79356:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571832.563049:0:79356:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571832.563133:0:79356:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d31e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 79356 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.563145:0:79356:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d31e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79356 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.563157:0:79356:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d31e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79356 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571832.563184:0:79356:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0700 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:79356:1875449613585408:192.168.202.113@tcp:101:createmany.0 00000100:00100000:0.0:1788571832.563282:0:79356:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada44fdf600 going to sleep for 11 seconds 00000100:00100000:0.0:1788571832.573670:0:79356:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0700 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:79356:1875449613585408:192.168.202.113@tcp:101:createmany.0 00010000:00010000:0.0:1788571832.573706:0:79356:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d31e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f70c0 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.573722:0:79356:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d31e lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f70c0 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.573748:0:79356:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d31e lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f70c0 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.573762:0:79356:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d31e lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f70c0 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.573778:0:79356:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d31e lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f70c0 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.573788:0:79356:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d31e lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f70c0 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.573796:0:79356:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d31e lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f70c0 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788571832.573808:0:79356:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada516f0700 x1875449613585408/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788571848 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788571832.573823:0:79356:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d31e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f70c0 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.573840:0:79356:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1788571832.573849:0:79356:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1788571832.573934:0:79356:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:0.0:1788571832.573942:0:79356:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d31e 00000080:00010000:0.0:1788571832.573949:0:79356:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571832.573954:0:79356:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d31e from it ffffab0ec1cbfbf8 00010000:00010000:0.0:1788571832.573957:0:79356:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d31e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f70c0 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.573966:0:79356:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d31e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f70c0 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571832.574006:0:79356:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:0.0:1788571832.575153:0:79356:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571832.575197:0:79356:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571832.575216:0:79356:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d31e lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f70c0 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571832.575233:0:79356:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d31e lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f70c0 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571832.575241:0:79356:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788571832.575251:0:79356:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571832.575255:0:79356:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d31e 00000080:00010000:0.0:1788571832.575262:0:79356:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d31e from it ffffab0ec1cbfb38 00010000:00010000:0.0:1788571832.575266:0:79356:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d31e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f70c0 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571832.575275:0:79356:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d31e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f70c0 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788571832.575290:0:79356:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:0.0:1788571832.575298:0:79356:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33b.replay-single--5, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), file ffff9ada600b2c00, open_flags 8042, mode 81a4 opened 0 00000080:00200000:0.0:1788571832.575308:0:79356:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33b.replay-single--5, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000002:00010000:0.0:1788571832.575336:0:79356:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33b.replay-single--5,[0x200016b60:0x6:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:0.0:1788571832.576159:0:79356:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d31e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b82f70c0 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571832.576244:0:79356:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d31e lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f70c0 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571832.576253:0:79356:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1788571832.576256:0:79356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d31e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f70c0 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571832.576306:0:79356:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d325 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 79356 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.576315:0:79356:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d325 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79356 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:0.0:1788571832.576330:0:79356:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:0.0:1788571832.577214:0:79356:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada516f0700 x1875449613585664/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:0.0:1788571832.577233:0:79356:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d325 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79356 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571832.577269:0:79356:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0700 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:79356:1875449613585664:192.168.202.113@tcp:101:createmany.0 00000100:00100000:0.0:1788571832.577475:0:79356:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada44fdf600 going to sleep for 11 seconds 00000100:00100000:0.0:1788571832.594078:0:79356:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0700 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:79356:1875449613585664:192.168.202.113@tcp:101:createmany.0 00010000:00010000:0.0:1788571832.594123:0:79356:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d325 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f70dc expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.594164:0:79356:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d325 lrc: 4/0,1 mode: --/PR res: [0x200016b60:0x6:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f70dc expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.594174:0:79356:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d325 lrc: 4/0,1 mode: --/PR res: [0x200016b60:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f70dc expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.594188:0:79356:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d325 lrc: 4/0,1 mode: PR/PR res: [0x200016b60:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f70dc expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.594206:0:79356:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d325 lrc: 4/0,1 mode: PR/PR res: [0x200016b60:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f70dc expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.594222:0:79356:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d325 lrc: 5/1,1 mode: PR/PR res: [0x200016b60:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f70dc expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.594231:0:79356:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d325 lrc: 5/1,1 mode: PR/PR res: [0x200016b60:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f70dc expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.594239:0:79356:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d325 lrc: 4/1,0 mode: PR/PR res: [0x200016b60:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f70dc expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788571832.594253:0:79356:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9ada516f0700 x1875449613585664/t197568495627(197568495627) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/664 e 0 to 0 dl 1788571848 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00100000:0.0:1788571832.594267:0:79356:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada516f0700 x1875449613585664/t197568495627(197568495627) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/664 e 0 to 0 dl 1788571848 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788571832.594283:0:79356:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d325 lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f70dc expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:0.0:1788571832.594297:0:79356:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d325 lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f70dc expref: -99 pid: 79356 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.594310:0:79356:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8590027616/6 (0 0) 00000080:00200000:0.0:1788571832.594318:0:79356:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1788571832.594336:0:79356:0:(namei.c:1628:ll_create_it()) VFS Op:name=f33b.replay-single--5, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000080:00200000:0.0:1788571832.594383:0:79356:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200016b60:0x6:0x0](ffff9ada58215b88) 00000080:00200000:0.0:1788571832.594555:0:79356:0:(file.c:6843:ll_layout_conf()) [0x200016b60:0x6:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:0.0:1788571832.594561:0:79356:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200016b60:0x6:0x0](ffff9ada58215b88) 00000080:00010000:0.0:1788571832.594564:0:79356:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200016b60:0x6:0x0](ffff9ada58215b88) for lock 0xef30c340bfb3d325 00000080:00010000:0.0:1788571832.594576:0:79356:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200016b60:0x6:0x0](ffff9ada58215b88) 00000080:00200008:0.0:1788571832.594595:0:79356:0:(file.c:1090:ll_file_open()) START file f33b.replay-single--5:[0x200016b60:0x6:0x0](ffff9ada58215b88), flags 100102 00000002:00100000:0.0:1788571832.594619:0:79356:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada516f0700 x1875449613585664/t197568495627(197568495627) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571848 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:0.0:1788571832.594661:0:79356:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d325 from it ffff9ada51761cc0 00010000:00010000:0.0:1788571832.594667:0:79356:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d325 lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f70dc expref: -99 pid: 79356 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.594677:0:79356:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d325 lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f70dc expref: -99 pid: 79356 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571832.594703:0:79356:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200016b60:0x6:0x0](ffff9ada58215b88), inode mode 81a4 mask 40 00000080:00200008:0.0:1788571832.594754:0:79356:0:(file.c:479:ll_file_release()) START file f33b.replay-single--5:[0x200016b60:0x6:0x0](ffff9ada58215b88), flags 100002 00000002:00100000:0.0:1788571832.594829:0:79356:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9ada516f0700 x1875449613585664/t197568495627(197568495627) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571848 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:0.0:1788571832.594840:0:79356:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:0.0:1788571832.594846:0:79356:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada516f0e00 x1875449613585920/t0(0) o35->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:0.0:1788571832.594954:0:79356:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0e00 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:79356:1875449613585920:192.168.202.113@tcp:35:createmany.0 00000100:00100000:0.0:1788571832.595054:0:79356:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada44fdf780 going to sleep for 11 seconds 00000080:00200000:2.0:1788571832.631344:0:79356:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200016b60:0x8:0x0](ffff9ada581942c8) 00000080:00200000:2.0:1788571832.631455:0:79356:0:(file.c:6843:ll_layout_conf()) [0x200016b60:0x8:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1788571832.631461:0:79356:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200016b60:0x8:0x0](ffff9ada581942c8) 00000080:00010000:2.0:1788571832.631466:0:79356:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200016b60:0x8:0x0](ffff9ada581942c8) for lock 0xef30c340bfb3d341 00000080:00010000:2.0:1788571832.631479:0:79356:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200016b60:0x8:0x0](ffff9ada581942c8) 00000080:00200008:2.0:1788571832.631500:0:79356:0:(file.c:1090:ll_file_open()) START file f33b.replay-single--7:[0x200016b60:0x8:0x0](ffff9ada581942c8), flags 100102 00000002:00100000:2.0:1788571832.631527:0:79356:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada603f6d80 x1875449613586944/t197568495631(197568495631) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571848 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1788571832.631583:0:79356:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d341 from it ffff9ada60a45ba0 00010000:00010000:2.0:1788571832.631590:0:79356:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f200/0xef30c340bfb3d341 lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f713e expref: -99 pid: 79356 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.631600:0:79356:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f200/0xef30c340bfb3d341 lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f713e expref: -99 pid: 79356 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571832.631628:0:79356:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200016b60:0x8:0x0](ffff9ada581942c8), inode mode 81a4 mask 40 00000080:00200008:2.0:1788571832.631669:0:79356:0:(file.c:479:ll_file_release()) START file f33b.replay-single--7:[0x200016b60:0x8:0x0](ffff9ada581942c8), flags 100002 00000002:00100000:2.0:1788571832.631726:0:79356:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9ada603f6d80 x1875449613586944/t197568495631(197568495631) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571848 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1788571832.631750:0:79356:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1788571832.631756:0:79356:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada60a34a80 x1875449613587200/t0(0) o35->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1788571832.631783:0:79356:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a34a80 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:79356:1875449613587200:192.168.202.113@tcp:35:createmany.0 00000100:00100000:2.0:1788571832.631921:0:79356:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6059bc00 going to sleep for 11 seconds 00000100:00100000:0.0:1788571832.636343:0:79356:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a34a80 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:79356:1875449613587200:192.168.202.113@tcp:35:createmany.0 00000002:00100000:0.0:1788571832.636410:0:79356:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9ada603f6d80 x1875449613586944/t197568495631(197568495631) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571848 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'createmany.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1788571832.636513:0:79356:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571832.636539:0:79356:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571832.636631:0:79356:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d348 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 79356 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.636643:0:79356:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d348 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79356 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.636656:0:79356:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d348 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79356 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571832.636683:0:79356:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f2300 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:79356:1875449613587328:192.168.202.113@tcp:101:createmany.0 00000100:00100000:0.0:1788571832.636792:0:79356:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada44fdf780 going to sleep for 11 seconds 00000100:00100000:0.0:1788571832.641200:0:79356:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f2300 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:79356:1875449613587328:192.168.202.113@tcp:101:createmany.0 00010000:00010000:0.0:1788571832.641240:0:79356:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d348 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7153 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.641254:0:79356:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d348 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7153 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.641269:0:79356:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d348 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7153 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.641283:0:79356:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d348 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7153 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.641303:0:79356:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d348 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7153 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.641312:0:79356:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d348 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7153 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.641321:0:79356:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d348 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7153 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788571832.641334:0:79356:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada516f2300 x1875449613587328/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788571848 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'createmany.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788571832.641414:0:79356:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d348 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7153 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.641432:0:79356:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1788571832.641441:0:79356:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1788571832.641480:0:79356:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:0.0:1788571832.641488:0:79356:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d348 00000080:00010000:0.0:1788571832.641494:0:79356:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571832.641498:0:79356:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d348 from it ffffab0ec1cbfbf8 00010000:00010000:0.0:1788571832.641502:0:79356:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d348 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7153 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.641511:0:79356:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d348 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7153 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571832.641548:0:79356:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:0.0:1788571832.641706:0:79356:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571832.641738:0:79356:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571832.641749:0:79356:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d348 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7153 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571832.641762:0:79356:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d348 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7153 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571832.641769:0:79356:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788571832.641774:0:79356:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571832.641776:0:79356:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d348 00000080:00010000:0.0:1788571832.641780:0:79356:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d348 from it ffffab0ec1cbfb38 00010000:00010000:0.0:1788571832.641783:0:79356:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d348 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7153 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571832.641791:0:79356:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d348 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7153 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788571832.643631:0:79356:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:0.0:1788571832.643653:0:79356:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f33b.replay-single--8, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), file ffff9ada600b3380, open_flags 8042, mode 81a4 opened 0 00000080:00200000:0.0:1788571832.643672:0:79356:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f33b.replay-single--8, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000002:00010000:0.0:1788571832.643712:0:79356:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f33b.replay-single--8,[0x200016b60:0x9:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:0.0:1788571832.643747:0:79356:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d348 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b82f7153 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571832.644333:0:79356:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d348 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f7153 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571832.644344:0:79356:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1788571832.644348:0:79356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d348 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f7153 expref: -99 pid: 79356 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571832.644470:0:79356:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617000/0xef30c340bfb3d34f lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 79356 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.644479:0:79356:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617000/0xef30c340bfb3d34f lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79356 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:0.0:1788571832.644493:0:79356:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:0.0:1788571832.644510:0:79356:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada516f2300 x1875449613587584/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:0.0:1788571832.644521:0:79356:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617000/0xef30c340bfb3d34f lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79356 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571832.644550:0:79356:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f2300 pname:cluuid:pid:xid:nid:opc:job createmany:35eb2d60-2de0-4b00-9815-d5153edd0057:79356:1875449613587584:192.168.202.113@tcp:101:createmany.0 00000100:00100000:0.0:1788571832.644967:0:79356:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada44fdf780 going to sleep for 11 seconds 00000080:00200000:2.0:1788571832.682576:0:78627:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571832.682616:0:78627:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571832.682716:0:78627:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c600/0xef30c340bfb3d364 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 78627 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.682739:0:78627:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c600/0xef30c340bfb3d364 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78627 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.682751:0:78627:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c600/0xef30c340bfb3d364 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 78627 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571832.682786:0:78627:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a34700 pname:cluuid:pid:xid:nid:opc:job bash:35eb2d60-2de0-4b00-9815-d5153edd0057:78627:1875449613588608:192.168.202.113@tcp:101:bash.0 00000100:00100000:2.0:1788571832.682943:0:78627:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6059bd80 going to sleep for 11 seconds 00000100:00100000:0.0:1788571832.686775:0:78627:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a34700 pname:cluuid:pid:xid:nid:opc:job bash:35eb2d60-2de0-4b00-9815-d5153edd0057:78627:1875449613588608:192.168.202.113@tcp:101:bash.0 00010000:00010000:0.0:1788571832.686824:0:78627:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c600/0xef30c340bfb3d364 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f71b5 expref: -99 pid: 78627 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.686839:0:78627:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c600/0xef30c340bfb3d364 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f71b5 expref: -99 pid: 78627 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.686892:0:78627:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c600/0xef30c340bfb3d364 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f71b5 expref: -99 pid: 78627 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.686906:0:78627:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c600/0xef30c340bfb3d364 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f71b5 expref: -99 pid: 78627 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.686922:0:78627:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c600/0xef30c340bfb3d364 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f71b5 expref: -99 pid: 78627 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.686931:0:78627:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c600/0xef30c340bfb3d364 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f71b5 expref: -99 pid: 78627 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.686939:0:78627:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c600/0xef30c340bfb3d364 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f71b5 expref: -99 pid: 78627 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788571832.686952:0:78627:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada60a34700 x1875449613588608/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788571848 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788571832.686969:0:78627:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c600/0xef30c340bfb3d364 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f71b5 expref: -99 pid: 78627 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.686995:0:78627:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1788571832.687002:0:78627:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1788571832.687268:0:78627:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:0.0:1788571832.687279:0:78627:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d364 00000080:00010000:0.0:1788571832.687287:0:78627:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571832.687291:0:78627:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d364 from it ffffab0ec1bbfbb0 00010000:00010000:0.0:1788571832.687297:0:78627:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c600/0xef30c340bfb3d364 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f71b5 expref: -99 pid: 78627 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.687307:0:78627:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c600/0xef30c340bfb3d364 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f71b5 expref: -99 pid: 78627 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571832.687338:0:78627:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:0.0:1788571832.687369:0:78627:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 304000 00000080:00200008:0.0:1788571832.687401:0:78627:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000002:00010000:0.0:1788571832.687432:0:78627:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: readdir flags 0 00010000:00010000:0.0:1788571832.687442:0:78627:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c600/0xef30c340bfb3d364 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f71b5 expref: -99 pid: 78627 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571832.687454:0:78627:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c600/0xef30c340bfb3d364 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f71b5 expref: -99 pid: 78627 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000100:00100000:0.0:1788571832.688513:0:78627:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f1500 pname:cluuid:pid:xid:nid:opc:job bash:35eb2d60-2de0-4b00-9815-d5153edd0057:78627:1875449613588736:192.168.202.113@tcp:37:bash.0 00000100:00100000:0.0:1788571832.689218:0:78627:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada44fdf240 going to sleep for 11 seconds 00000100:00100000:0.0:1788571832.697034:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613588992:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571832.701064:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613589248:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571832.714817:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449613588864:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571832.714848:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613589120:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571832.714928:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613588992:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571832.715886:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613589376:192.168.202.113@tcp:101:rm.0 00010000:00010000:0.0:1788571832.720341:0:2383:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 0000000066164d3b/0xef30c340bfb3d36b lrc: 6/2,0 mode: --/PR res: [0x280000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.720427:0:2383:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 0000000066164d3b/0xef30c340bfb3d36b lrc: 6/2,0 mode: --/PR res: [0x280000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f71bc expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.720447:0:2383:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 0000000066164d3b/0xef30c340bfb3d36b lrc: 6/2,0 mode: PR/PR res: [0x280000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f71bc expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.720461:0:2383:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 0000000066164d3b/0xef30c340bfb3d36b lrc: 6/2,0 mode: PR/PR res: [0x280000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f71bc expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1788571832.720479:0:2383:0:(osc_lock.c:187:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 0000000066164d3b/0xef30c340bfb3d36b lrc: 5/2,0 mode: PR/PR res: [0x280000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f71bc expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:0.0:1788571832.720494:0:2383:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00010000:00010000:0.0:1788571832.720547:0:2383:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 0000000066164d3b/0xef30c340bfb3d36b lrc: 6/2,0 mode: PR/PR res: [0x280000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0xdb7edb52b82f71bc expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.720559:0:2383:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 0000000066164d3b/0xef30c340bfb3d36b lrc: 5/1,0 mode: PR/PR res: [0x280000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0xdb7edb52b82f71bc expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.720636:0:2383:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 0000000066164d3b/0xef30c340bfb3d36b lrc: 5/1,0 mode: PR/PR res: [0x280000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0xdb7edb52b82f71bc expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.720653:0:2383:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 0000000066164d3b/0xef30c340bfb3d36b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0xdb7edb52b82f71bc expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:0.0:1788571832.720674:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613589376:192.168.202.113@tcp:101:rm.0 00000100:00100000:0.0:1788571832.735177:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613589888:192.168.202.113@tcp:101:rm.0 00010000:00010000:0.0:1788571832.742230:0:2384:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 000000003d4c2635/0xef30c340bfb3d379 lrc: 6/2,0 mode: --/PR res: [0x2c0000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.742271:0:2384:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 000000003d4c2635/0xef30c340bfb3d379 lrc: 6/2,0 mode: --/PR res: [0x2c0000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f71ed expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.742291:0:2384:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 000000003d4c2635/0xef30c340bfb3d379 lrc: 6/2,0 mode: PR/PR res: [0x2c0000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f71ed expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.742307:0:2384:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 000000003d4c2635/0xef30c340bfb3d379 lrc: 6/2,0 mode: PR/PR res: [0x2c0000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f71ed expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1788571832.742325:0:2384:0:(osc_lock.c:187:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 000000003d4c2635/0xef30c340bfb3d379 lrc: 5/2,0 mode: PR/PR res: [0x2c0000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f71ed expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:0.0:1788571832.742338:0:2384:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00010000:00010000:0.0:1788571832.742428:0:2384:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 000000003d4c2635/0xef30c340bfb3d379 lrc: 6/2,0 mode: PR/PR res: [0x2c0000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0xdb7edb52b82f71ed expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.742446:0:2384:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 000000003d4c2635/0xef30c340bfb3d379 lrc: 5/1,0 mode: PR/PR res: [0x2c0000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0xdb7edb52b82f71ed expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.742455:0:2384:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 000000003d4c2635/0xef30c340bfb3d379 lrc: 5/1,0 mode: PR/PR res: [0x2c0000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0xdb7edb52b82f71ed expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.742465:0:2384:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 000000003d4c2635/0xef30c340bfb3d379 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x602:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0xdb7edb52b82f71ed expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:0.0:1788571832.742491:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613589888:192.168.202.113@tcp:101:rm.0 00000080:00200000:2.0:1788571832.742653:0:79357:0:(file.c:1775:ll_merge_attr_nolock()) [0x200016b60:0x2:0x0] updating i_size 0 i_blocks 0 00000020:00010000:2.0:1788571832.742661:0:79357:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9ada51687090 (ffff9ada60b68638/0) at cl_lock_release():245 00000020:00010000:2.0:1788571832.742664:0:79357:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9ada51687090 (ffff9ada60b68638/0) at cl_lock_cancel():142 00000020:00010000:2.0:1788571832.742670:0:79357:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9ada51899c30 (ffff9ada60b68488/1) at cl_lock_cancel():142 00000020:00010000:2.0:1788571832.742675:0:79357:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9ada51687090 (ffff9ada60b68638/0) at cl_lock_fini():72 00000020:00010000:2.0:1788571832.742678:0:79357:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9ada51899c30 (ffff9ada60b68638/1) at cl_lock_fini():72 00000080:00200000:2.0:1788571832.742723:0:79357:0:(vvp_io.c:304:vvp_io_fini()) [0x200016b60:0x2:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1788571832.742803:0:79357:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571832.742833:0:79357:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571832.742874:0:79357:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e200/0xef30c340bfb3d372 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f71e6 expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571832.742891:0:79357:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e200/0xef30c340bfb3d372 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f71e6 expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571832.742901:0:79357:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571832.742912:0:79357:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571832.742917:0:79357:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d372 00000080:00010000:2.0:1788571832.742924:0:79357:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d372 from it ffffab0ec1d37c10 00010000:00010000:2.0:1788571832.742928:0:79357:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e200/0xef30c340bfb3d372 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f71e6 expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571832.742936:0:79357:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e200/0xef30c340bfb3d372 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f71e6 expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571832.742948:0:79357:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:2.0:1788571832.742962:0:79357:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33b.replay-single--1, flags=0 00000080:00200000:2.0:1788571832.742971:0:79357:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571832.743112:0:79357:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571832.743121:0:79357:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e200/0xef30c340bfb3d372 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f71e6 expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571832.743131:0:79357:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e200/0xef30c340bfb3d372 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f71e6 expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571832.743138:0:79357:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571832.743143:0:79357:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571832.743146:0:79357:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d372 00000080:00010000:2.0:1788571832.743150:0:79357:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d372 from it ffffab0ec1d37d68 00010000:00010000:2.0:1788571832.743154:0:79357:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e200/0xef30c340bfb3d372 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f71e6 expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571832.743162:0:79357:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e200/0xef30c340bfb3d372 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f71e6 expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1788571832.743169:0:79357:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:2.0:1788571832.743176:0:79357:0:(namei.c:2351:ll_unlink()) VFS Op:name=f33b.replay-single--1, dir=[0x200000007:0x1:0x0](ffff9ada58125b88) 00010000:00010000:2.0:1788571832.743211:0:79357:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e200/0xef30c340bfb3d372 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b82f71e6 expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1788571832.743244:0:79357:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d2ed lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0xdb7edb52b82f7018 expref: -99 pid: 79356 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571832.743256:0:79357:0:(vvp_object.c:117:vvp_conf_set()) [0x200016b60:0x2:0x0]: losing layout lock 00010000:00010000:2.0:1788571832.743302:0:79357:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d2ed lrc: 1/0,0 mode: --/PR res: [0x200016b60:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0xdb7edb52b82f7018 expref: -99 pid: 79356 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571832.743315:0:79357:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e200/0xef30c340bfb3d372 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f71e6 expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1788571832.743321:0:79357:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:2.0:1788571832.743325:0:79357:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d2ed lrc: 0/0,0 mode: --/PR res: [0x200016b60:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0xdb7edb52b82f7018 expref: -99 pid: 79356 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571832.743547:0:79357:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e200/0xef30c340bfb3d372 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f71e6 expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:2.0:1788571832.743640:0:79357:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1788571832.743654:0:79357:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada60a35880 x1875449613590144/t0(0) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788571832.743684:0:79357:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35880 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:79357:1875449613590144:192.168.202.113@tcp:36:rm.0 00000100:00100000:2.0:1788571832.743889:0:79357:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6059b600 going to sleep for 11 seconds 00000100:00100000:0.0:1788571832.750425:0:79357:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a35880 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:79357:1875449613590144:192.168.202.113@tcp:36:rm.0 00000080:00200000:0.0:1788571832.750509:0:79357:0:(vvp_io.c:1826:vvp_io_init()) [0x200016b60:0x2:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1788571832.750529:0:79357:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1788571832.750684:0:79357:0:(vvp_io.c:304:vvp_io_fini()) [0x200016b60:0x2:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1788571832.750694:0:79357:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200016b60:0x2:0x0](ffff9ada58217448) 00000008:00010000:0.0:1788571832.750750:0:79357:0:(osc_object.c:202:osc_object_ast_clear()) ### update lock size 0 blocks 0 [cma]time: 0 0 0 by oinfo size 0 blocks 0 [cma]time 0 0 0 ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 000000003d4c2635/0xef30c340bfb3d379 lrc: 1/0,0 mode: PR/PR res: [0x2c0000401:0x602:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0xdb7edb52b82f71ed expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571832.750912:0:79357:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571832.750939:0:79357:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571832.751297:0:79357:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d380 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 79357 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.751314:0:79357:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d380 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79357 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.751325:0:79357:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d380 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79357 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571832.751421:0:79357:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f1c00 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:79357:1875449613590272:192.168.202.113@tcp:101:rm.0 00000100:00100000:0.0:1788571832.751575:0:79357:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada44fdff00 going to sleep for 11 seconds 00000100:00100000:0.0:1788571832.755764:0:79357:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f1c00 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:79357:1875449613590272:192.168.202.113@tcp:101:rm.0 00010000:00010000:0.0:1788571832.755845:0:79357:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d380 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7217 expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.755941:0:79357:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d380 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7217 expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.755958:0:79357:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d380 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7217 expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.755973:0:79357:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d380 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7217 expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.756112:0:79357:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d380 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7217 expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.756122:0:79357:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d380 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7217 expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.756136:0:79357:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d380 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7217 expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788571832.756151:0:79357:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada516f1c00 x1875449613590272/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788571848 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788571832.756170:0:79357:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617400/0xef30c340bfb3d380 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7217 expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.756188:0:79357:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1788571832.756197:0:79357:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000100:00100000:0.0:1788571832.757218:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613590400:192.168.202.113@tcp:101:rm.0 00010000:00010000:0.0:1788571832.760839:0:2383:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000a55255e2/0xef30c340bfb3d387 lrc: 6/2,0 mode: --/PR res: [0x280000401:0x603:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.760894:0:2383:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000a55255e2/0xef30c340bfb3d387 lrc: 6/2,0 mode: --/PR res: [0x280000401:0x603:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f721e expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.760914:0:2383:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000a55255e2/0xef30c340bfb3d387 lrc: 6/2,0 mode: PR/PR res: [0x280000401:0x603:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f721e expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.760929:0:2383:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000a55255e2/0xef30c340bfb3d387 lrc: 6/2,0 mode: PR/PR res: [0x280000401:0x603:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f721e expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1788571832.760948:0:2383:0:(osc_lock.c:187:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000a55255e2/0xef30c340bfb3d387 lrc: 5/2,0 mode: PR/PR res: [0x280000401:0x603:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f721e expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:0.0:1788571832.760963:0:2383:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00010000:00010000:0.0:1788571832.761138:0:2383:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000a55255e2/0xef30c340bfb3d387 lrc: 6/2,0 mode: PR/PR res: [0x280000401:0x603:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0xdb7edb52b82f721e expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.761149:0:2383:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000a55255e2/0xef30c340bfb3d387 lrc: 5/1,0 mode: PR/PR res: [0x280000401:0x603:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0xdb7edb52b82f721e expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.761157:0:2383:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000a55255e2/0xef30c340bfb3d387 lrc: 5/1,0 mode: PR/PR res: [0x280000401:0x603:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0xdb7edb52b82f721e expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.761167:0:2383:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000a55255e2/0xef30c340bfb3d387 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x603:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0xdb7edb52b82f721e expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:0.0:1788571832.761193:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613590400:192.168.202.113@tcp:101:rm.0 00000080:00200000:2.0:1788571832.778497:0:79357:0:(file.c:1775:ll_merge_attr_nolock()) [0x200016b60:0x4:0x0] updating i_size 0 i_blocks 0 00000020:00010000:2.0:1788571832.778505:0:79357:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9ada51685970 (ffff9ada60b68e18/0) at cl_lock_release():245 00000020:00010000:2.0:1788571832.778509:0:79357:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9ada51685970 (ffff9ada60b68e18/0) at cl_lock_cancel():142 00000020:00010000:2.0:1788571832.778514:0:79357:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9ada51899cb0 (ffff9ada60b68a28/1) at cl_lock_cancel():142 00000020:00010000:2.0:1788571832.778518:0:79357:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9ada51685970 (ffff9ada60b68e18/0) at cl_lock_fini():72 00000020:00010000:2.0:1788571832.778520:0:79357:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9ada51899cb0 (ffff9ada60b68e18/1) at cl_lock_fini():72 00000080:00200000:2.0:1788571832.778624:0:79357:0:(vvp_io.c:304:vvp_io_fini()) [0x200016b60:0x4:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1788571832.778693:0:79357:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571832.778720:0:79357:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571832.778749:0:79357:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e400/0xef30c340bfb3d38e lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7248 expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571832.778765:0:79357:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e400/0xef30c340bfb3d38e lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7248 expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571832.778774:0:79357:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571832.778783:0:79357:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571832.778787:0:79357:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d38e 00000080:00010000:2.0:1788571832.778795:0:79357:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d38e from it ffffab0ec1d37c10 00010000:00010000:2.0:1788571832.778938:0:79357:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e400/0xef30c340bfb3d38e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7248 expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571832.778949:0:79357:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e400/0xef30c340bfb3d38e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7248 expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571832.778961:0:79357:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:2.0:1788571832.778974:0:79357:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33b.replay-single--3, flags=0 00000080:00200000:2.0:1788571832.779092:0:79357:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571832.779102:0:79357:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571832.779109:0:79357:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e400/0xef30c340bfb3d38e lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7248 expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571832.779118:0:79357:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e400/0xef30c340bfb3d38e lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7248 expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571832.779125:0:79357:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571832.779129:0:79357:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571832.779132:0:79357:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d38e 00000080:00010000:2.0:1788571832.779136:0:79357:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d38e from it ffffab0ec1d37d68 00010000:00010000:2.0:1788571832.779140:0:79357:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e400/0xef30c340bfb3d38e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7248 expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571832.779148:0:79357:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e400/0xef30c340bfb3d38e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7248 expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1788571832.779157:0:79357:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:2.0:1788571832.779162:0:79357:0:(namei.c:2351:ll_unlink()) VFS Op:name=f33b.replay-single--3, dir=[0x200000007:0x1:0x0](ffff9ada58125b88) 00010000:00010000:2.0:1788571832.779196:0:79357:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e400/0xef30c340bfb3d38e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b82f7248 expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1788571832.779226:0:79357:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d309 lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0xdb7edb52b82f707a expref: -99 pid: 79356 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571832.779236:0:79357:0:(vvp_object.c:117:vvp_conf_set()) [0x200016b60:0x4:0x0]: losing layout lock 00010000:00010000:2.0:1788571832.779281:0:79357:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d309 lrc: 1/0,0 mode: --/PR res: [0x200016b60:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0xdb7edb52b82f707a expref: -99 pid: 79356 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571832.779288:0:79357:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e400/0xef30c340bfb3d38e lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f7248 expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1788571832.779295:0:79357:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:2.0:1788571832.779299:0:79357:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d309 lrc: 0/0,0 mode: --/PR res: [0x200016b60:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0xdb7edb52b82f707a expref: -99 pid: 79356 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571832.779323:0:79357:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e400/0xef30c340bfb3d38e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f7248 expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:2.0:1788571832.779349:0:79357:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1788571832.779420:0:79357:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada60a37b80 x1875449613591168/t0(0) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788571832.779454:0:79357:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37b80 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:79357:1875449613591168:192.168.202.113@tcp:36:rm.0 00000100:00100000:2.0:1788571832.779684:0:79357:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6059ba80 going to sleep for 11 seconds 00000100:00100000:2.0:1788571832.784266:0:79357:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37b80 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:79357:1875449613591168:192.168.202.113@tcp:36:rm.0 00000080:00200000:2.0:1788571832.784363:0:79357:0:(vvp_io.c:1826:vvp_io_init()) [0x200016b60:0x4:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1788571832.784383:0:79357:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1788571832.784422:0:79357:0:(vvp_io.c:304:vvp_io_fini()) [0x200016b60:0x4:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1788571832.784431:0:79357:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200016b60:0x4:0x0](ffff9ada5819ec08) 00000008:00010000:2.0:1788571832.784474:0:79357:0:(osc_object.c:202:osc_object_ast_clear()) ### update lock size 0 blocks 0 [cma]time: 0 0 0 by oinfo size 0 blocks 0 [cma]time 0 0 0 ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 00000000fd3161a4/0xef30c340bfb3d395 lrc: 1/0,0 mode: PR/PR res: [0x2c0000401:0x603:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0xdb7edb52b82f724f expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571832.784648:0:79357:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571832.784675:0:79357:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571832.784758:0:79357:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172dc00/0xef30c340bfb3d39c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 79357 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.784768:0:79357:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172dc00/0xef30c340bfb3d39c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79357 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.784779:0:79357:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172dc00/0xef30c340bfb3d39c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79357 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571832.784815:0:79357:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37800 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:79357:1875449613591296:192.168.202.113@tcp:101:rm.0 00000100:00100000:2.0:1788571832.784951:0:79357:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6059ba80 going to sleep for 11 seconds 00000100:00100000:2.0:1788571832.788226:0:79357:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37800 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:79357:1875449613591296:192.168.202.113@tcp:101:rm.0 00010000:00010000:2.0:1788571832.788267:0:79357:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172dc00/0xef30c340bfb3d39c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7279 expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.788281:0:79357:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172dc00/0xef30c340bfb3d39c lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7279 expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.788295:0:79357:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172dc00/0xef30c340bfb3d39c lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7279 expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.788308:0:79357:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172dc00/0xef30c340bfb3d39c lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7279 expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.788322:0:79357:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172dc00/0xef30c340bfb3d39c lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7279 expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.788331:0:79357:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172dc00/0xef30c340bfb3d39c lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7279 expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.788339:0:79357:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172dc00/0xef30c340bfb3d39c lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7279 expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1788571832.788364:0:79357:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada60a37800 x1875449613591296/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788571848 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1788571832.788381:0:79357:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172dc00/0xef30c340bfb3d39c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7279 expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.788397:0:79357:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:2.0:1788571832.788404:0:79357:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:2.0:1788571832.788441:0:79357:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:2.0:1788571832.788449:0:79357:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d39c 00000080:00010000:2.0:1788571832.788455:0:79357:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571832.788459:0:79357:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d39c from it ffffab0ec1d37b58 00010000:00010000:2.0:1788571832.788461:0:79357:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172dc00/0xef30c340bfb3d39c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7279 expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.788471:0:79357:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172dc00/0xef30c340bfb3d39c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7279 expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571832.788500:0:79357:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:2.0:1788571832.788514:0:79357:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33b.replay-single--4, flags=8 00000080:00200008:2.0:1788571832.788530:0:79357:0:(file.c:6296:ll_getattr_dentry()) START file f33b.replay-single--4:[0x200016b60:0x5:0x0](ffff9ada581980c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1788571832.788543:0:79357:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200016b60:0x5:0x0](ffff9ada581980c8),name=f33b.replay-single--4 00000002:00010000:2.0:1788571832.788583:0:79357:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200016b60:0x5:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571832.788597:0:79357:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d317 lrc: 2/1,0 mode: PR/PR res: [0x200016b60:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f70ab expref: -99 pid: 79356 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571832.788609:0:79357:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d317 lrc: 2/1,0 mode: PR/PR res: [0x200016b60:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f70ab expref: -99 pid: 79356 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571832.788617:0:79357:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571832.788622:0:79357:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200016b60:0x5:0x0](ffff9ada581980c8) 00000080:00010000:2.0:1788571832.788625:0:79357:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200016b60:0x5:0x0](ffff9ada581980c8) for lock 0xef30c340bfb3d317 00000080:00010000:2.0:1788571832.788629:0:79357:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d317 from it ffffab0ec1d37cb8 00010000:00010000:2.0:1788571832.788633:0:79357:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d317 lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f70ab expref: -99 pid: 79356 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571832.788645:0:79357:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d317 lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f70ab expref: -99 pid: 79356 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571832.788668:0:79357:0:(vvp_io.c:1826:vvp_io_init()) [0x200016b60:0x5:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:2.0:1788571832.788680:0:79357:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x200016b60:0x5:0x0] 00000008:00010000:2.0:1788571832.788706:0:79357:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a 00000008:00010000:2.0:1788571832.788710:0:79357:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff9ada48b9f8b0, osc lock ffff9ada4880fdb0, flags 0x400001002 00010000:00010000:2.0:1788571832.788783:0:79357:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 0000000055f784a0/0xef30c340bfb3d3a3 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x604:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.788796:0:79357:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x400001000 ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 0000000055f784a0/0xef30c340bfb3d3a3 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x604:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:2.0:1788571832.788842:0:79357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00000100:00100000:2.0:1788571832.788882:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1788571832.788904:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613591424:192.168.202.113@tcp:101:rm.0 00010000:00010000:2.0:1788571832.792889:0:2386:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 0000000055f784a0/0xef30c340bfb3d3a3 lrc: 6/2,0 mode: --/PR res: [0x280000401:0x604:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.792930:0:2386:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 0000000055f784a0/0xef30c340bfb3d3a3 lrc: 6/2,0 mode: --/PR res: [0x280000401:0x604:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7280 expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.792948:0:2386:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 0000000055f784a0/0xef30c340bfb3d3a3 lrc: 6/2,0 mode: PR/PR res: [0x280000401:0x604:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7280 expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.792965:0:2386:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 0000000055f784a0/0xef30c340bfb3d3a3 lrc: 6/2,0 mode: PR/PR res: [0x280000401:0x604:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7280 expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1788571832.793101:0:2386:0:(osc_lock.c:187:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 0000000055f784a0/0xef30c340bfb3d3a3 lrc: 5/2,0 mode: PR/PR res: [0x280000401:0x604:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f7280 expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:2.0:1788571832.793117:0:2386:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00010000:00010000:2.0:1788571832.793198:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 0000000055f784a0/0xef30c340bfb3d3a3 lrc: 6/2,0 mode: PR/PR res: [0x280000401:0x604:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0xdb7edb52b82f7280 expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.793209:0:2386:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 0000000055f784a0/0xef30c340bfb3d3a3 lrc: 5/1,0 mode: PR/PR res: [0x280000401:0x604:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0xdb7edb52b82f7280 expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.793216:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 0000000055f784a0/0xef30c340bfb3d3a3 lrc: 5/1,0 mode: PR/PR res: [0x280000401:0x604:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0xdb7edb52b82f7280 expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.793226:0:2386:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 0000000055f784a0/0xef30c340bfb3d3a3 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x604:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0xdb7edb52b82f7280 expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:2.0:1788571832.793249:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613591424:192.168.202.113@tcp:101:rm.0 00000080:00200000:2.0:1788571832.811550:0:79357:0:(file.c:1775:ll_merge_attr_nolock()) [0x200016b60:0x6:0x0] updating i_size 0 i_blocks 0 00000020:00010000:2.0:1788571832.811559:0:79357:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9ada60bc39d0 (ffff9ada60b0d248/0) at cl_lock_release():245 00000020:00010000:2.0:1788571832.811563:0:79357:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9ada60bc39d0 (ffff9ada60b0d248/0) at cl_lock_cancel():142 00000020:00010000:2.0:1788571832.811580:0:79357:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9ada51899fb0 (ffff9ada60b0d098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1788571832.811585:0:79357:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9ada60bc39d0 (ffff9ada60b0d248/0) at cl_lock_fini():72 00000020:00010000:2.0:1788571832.811589:0:79357:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9ada51899fb0 (ffff9ada60b0d248/1) at cl_lock_fini():72 00000080:00200000:2.0:1788571832.811605:0:79357:0:(vvp_io.c:304:vvp_io_fini()) [0x200016b60:0x6:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1788571832.811669:0:79357:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571832.811701:0:79357:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571832.811722:0:79357:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f600/0xef30c340bfb3d3aa lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f72aa expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571832.811755:0:79357:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f600/0xef30c340bfb3d3aa lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f72aa expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571832.811766:0:79357:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571832.811776:0:79357:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571832.811780:0:79357:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d3aa 00000080:00010000:2.0:1788571832.811788:0:79357:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d3aa from it ffffab0ec1d37c10 00010000:00010000:2.0:1788571832.811791:0:79357:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f600/0xef30c340bfb3d3aa lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f72aa expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571832.811800:0:79357:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f600/0xef30c340bfb3d3aa lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f72aa expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571832.811811:0:79357:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:2.0:1788571832.811824:0:79357:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33b.replay-single--5, flags=0 00000080:00200000:2.0:1788571832.811832:0:79357:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571832.811838:0:79357:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571832.811845:0:79357:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f600/0xef30c340bfb3d3aa lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f72aa expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571832.811902:0:79357:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f600/0xef30c340bfb3d3aa lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f72aa expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571832.811916:0:79357:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571832.811921:0:79357:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571832.811923:0:79357:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d3aa 00000080:00010000:2.0:1788571832.811926:0:79357:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d3aa from it ffffab0ec1d37d68 00010000:00010000:2.0:1788571832.811929:0:79357:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f600/0xef30c340bfb3d3aa lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f72aa expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571832.811936:0:79357:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f600/0xef30c340bfb3d3aa lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f72aa expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1788571832.811944:0:79357:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:2.0:1788571832.811950:0:79357:0:(namei.c:2351:ll_unlink()) VFS Op:name=f33b.replay-single--5, dir=[0x200000007:0x1:0x0](ffff9ada58125b88) 00010000:00010000:2.0:1788571832.811984:0:79357:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f600/0xef30c340bfb3d3aa lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b82f72aa expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1788571832.812015:0:79357:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d325 lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0xdb7edb52b82f70dc expref: -99 pid: 79356 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571832.812025:0:79357:0:(vvp_object.c:117:vvp_conf_set()) [0x200016b60:0x6:0x0]: losing layout lock 00010000:00010000:2.0:1788571832.812232:0:79357:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d325 lrc: 1/0,0 mode: --/PR res: [0x200016b60:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0xdb7edb52b82f70dc expref: -99 pid: 79356 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571832.812241:0:79357:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f600/0xef30c340bfb3d3aa lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f72aa expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1788571832.812246:0:79357:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:2.0:1788571832.812249:0:79357:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616000/0xef30c340bfb3d325 lrc: 0/0,0 mode: --/PR res: [0x200016b60:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0xdb7edb52b82f70dc expref: -99 pid: 79356 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571832.812271:0:79357:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f600/0xef30c340bfb3d3aa lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f72aa expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:2.0:1788571832.812293:0:79357:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1788571832.812305:0:79357:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada60a37800 x1875449613592192/t0(0) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788571832.812334:0:79357:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37800 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:79357:1875449613592192:192.168.202.113@tcp:36:rm.0 00000100:00100000:2.0:1788571832.812502:0:79357:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6059ba80 going to sleep for 11 seconds 00000100:00100000:2.0:1788571832.816532:0:79357:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37800 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:79357:1875449613592192:192.168.202.113@tcp:36:rm.0 00000080:00200000:2.0:1788571832.816686:0:79357:0:(vvp_io.c:1826:vvp_io_init()) [0x200016b60:0x6:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1788571832.816704:0:79357:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1788571832.816755:0:79357:0:(vvp_io.c:304:vvp_io_fini()) [0x200016b60:0x6:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1788571832.816765:0:79357:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200016b60:0x6:0x0](ffff9ada58215b88) 00000008:00010000:2.0:1788571832.816820:0:79357:0:(osc_object.c:202:osc_object_ast_clear()) ### update lock size 0 blocks 0 [cma]time: 0 0 0 by oinfo size 0 blocks 0 [cma]time 0 0 0 ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 000000007aea920d/0xef30c340bfb3d3b1 lrc: 1/0,0 mode: PR/PR res: [0x2c0000401:0x604:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0xdb7edb52b82f72b1 expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571832.817197:0:79357:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571832.817225:0:79357:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571832.817299:0:79357:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d3b8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 79357 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.817309:0:79357:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d3b8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79357 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.817321:0:79357:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d3b8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79357 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571832.817350:0:79357:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35f80 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:79357:1875449613592320:192.168.202.113@tcp:101:rm.0 00000100:00100000:2.0:1788571832.817796:0:79357:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6059ba80 going to sleep for 11 seconds 00000100:00100000:2.0:1788571832.822137:0:79357:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a35f80 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:79357:1875449613592320:192.168.202.113@tcp:101:rm.0 00010000:00010000:2.0:1788571832.822176:0:79357:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d3b8 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f72db expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.822191:0:79357:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d3b8 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f72db expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.822206:0:79357:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d3b8 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f72db expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.822218:0:79357:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d3b8 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f72db expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.822233:0:79357:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d3b8 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f72db expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.822243:0:79357:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d3b8 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f72db expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.822251:0:79357:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d3b8 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f72db expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1788571832.822263:0:79357:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada60a35f80 x1875449613592320/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788571848 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1788571832.822286:0:79357:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d3b8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f72db expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.822302:0:79357:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:2.0:1788571832.822308:0:79357:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:2.0:1788571832.822340:0:79357:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:2.0:1788571832.822346:0:79357:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d3b8 00000080:00010000:2.0:1788571832.822352:0:79357:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571832.822366:0:79357:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d3b8 from it ffffab0ec1d37b58 00010000:00010000:2.0:1788571832.822370:0:79357:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d3b8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f72db expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.822380:0:79357:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d3b8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f72db expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571832.822409:0:79357:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:2.0:1788571832.822424:0:79357:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33b.replay-single--6, flags=8 00000080:00200008:2.0:1788571832.822438:0:79357:0:(file.c:6296:ll_getattr_dentry()) START file f33b.replay-single--6:[0x200016b60:0x7:0x0](ffff9ada58253248), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1788571832.824178:0:79357:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200016b60:0x7:0x0](ffff9ada58253248),name=f33b.replay-single--6 00000002:00010000:2.0:1788571832.824225:0:79357:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200016b60:0x7:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571832.824248:0:79357:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f000/0xef30c340bfb3d333 lrc: 2/1,0 mode: PR/PR res: [0x200016b60:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f710d expref: -99 pid: 79356 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571832.824265:0:79357:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f000/0xef30c340bfb3d333 lrc: 2/1,0 mode: PR/PR res: [0x200016b60:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f710d expref: -99 pid: 79356 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571832.824275:0:79357:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571832.824287:0:79357:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200016b60:0x7:0x0](ffff9ada58253248) 00000080:00010000:2.0:1788571832.824291:0:79357:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200016b60:0x7:0x0](ffff9ada58253248) for lock 0xef30c340bfb3d333 00000080:00010000:2.0:1788571832.824300:0:79357:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d333 from it ffffab0ec1d37cb8 00010000:00010000:2.0:1788571832.824304:0:79357:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f000/0xef30c340bfb3d333 lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f710d expref: -99 pid: 79356 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571832.824315:0:79357:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f000/0xef30c340bfb3d333 lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f710d expref: -99 pid: 79356 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571832.824349:0:79357:0:(vvp_io.c:1826:vvp_io_init()) [0x200016b60:0x7:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000100:00100000:0.0:1788571832.826080:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613592448:192.168.202.113@tcp:63:ll_sa.0 00000100:00100000:0.0:1788571832.826127:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613592832:192.168.202.113@tcp:101:ll_sa.0 00000100:00100000:0.0:1788571832.826152:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1788571832.826159:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613592576:192.168.202.113@tcp:101:ll_sa.0 00000001:00010000:2.0:1788571832.826525:0:79357:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x200016b60:0x7:0x0] 00000008:00010000:2.0:1788571832.826539:0:79357:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a 00000008:00010000:2.0:1788571832.826543:0:79357:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff9ada48b9ffb0, osc lock ffff9ada4880eab0, flags 0x400001002 00010000:00010000:2.0:1788571832.827329:0:79357:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 000000003829773f/0xef30c340bfb3d3f7 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x605:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.827346:0:79357:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x400001000 ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 000000003829773f/0xef30c340bfb3d3f7 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x605:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 79357 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:2.0:1788571832.827696:0:79357:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00010000:00010000:0.0:1788571832.832757:0:2384:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e200/0xef30c340bfb3d3bf lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.832782:0:2384:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e200/0xef30c340bfb3d3bf lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.832792:0:2384:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e200/0xef30c340bfb3d3bf lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.832799:0:2384:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e200/0xef30c340bfb3d3bf lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.832809:0:2384:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e200/0xef30c340bfb3d3bf lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.832816:0:2384:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada5163e200 already unused, calling callback (ffffffffc1966630) 00010000:00010000:0.0:1788571832.832821:0:2384:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e200/0xef30c340bfb3d3bf lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 7 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.832829:0:2384:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e200/0xef30c340bfb3d3bf lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 7 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.832837:0:2384:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e200/0xef30c340bfb3d3bf lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 7 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.832845:0:2384:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1788571832.832889:0:2384:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e200/0xef30c340bfb3d3bf lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 7 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.832897:0:2384:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163e200/0xef30c340bfb3d3bf lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 7 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.832931:0:2384:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f600/0xef30c340bfb3d3c6 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.832938:0:2384:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f600/0xef30c340bfb3d3c6 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.832949:0:2384:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f600/0xef30c340bfb3d3c6 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.832954:0:2384:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f600/0xef30c340bfb3d3c6 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.832960:0:2384:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f600/0xef30c340bfb3d3c6 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.832965:0:2384:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada5163f600 already unused, calling callback (ffffffffc1966630) 00010000:00010000:0.0:1788571832.832968:0:2384:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f600/0xef30c340bfb3d3c6 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 6 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.832973:0:2384:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f600/0xef30c340bfb3d3c6 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 6 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.832979:0:2384:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f600/0xef30c340bfb3d3c6 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 6 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.833153:0:2384:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1788571832.833158:0:2384:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f600/0xef30c340bfb3d3c6 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 6 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.833165:0:2384:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f600/0xef30c340bfb3d3c6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 6 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.833181:0:2384:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017000/0xef30c340bfb3d3cd lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.833188:0:2384:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017000/0xef30c340bfb3d3cd lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.833198:0:2384:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017000/0xef30c340bfb3d3cd lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.833204:0:2384:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017000/0xef30c340bfb3d3cd lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.833210:0:2384:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017000/0xef30c340bfb3d3cd lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.833216:0:2384:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada53017000 already unused, calling callback (ffffffffc1966630) 00010000:00010000:0.0:1788571832.833219:0:2384:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017000/0xef30c340bfb3d3cd lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 5 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.833225:0:2384:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017000/0xef30c340bfb3d3cd lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 5 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.833230:0:2384:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017000/0xef30c340bfb3d3cd lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 5 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.833236:0:2384:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1788571832.833240:0:2384:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017000/0xef30c340bfb3d3cd lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 5 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.833246:0:2384:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017000/0xef30c340bfb3d3cd lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 5 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.833258:0:2384:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016a00/0xef30c340bfb3d3d4 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.833264:0:2384:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016a00/0xef30c340bfb3d3d4 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.833273:0:2384:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016a00/0xef30c340bfb3d3d4 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.833279:0:2384:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016a00/0xef30c340bfb3d3d4 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.833285:0:2384:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016a00/0xef30c340bfb3d3d4 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.833289:0:2384:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada53016a00 already unused, calling callback (ffffffffc1966630) 00010000:00010000:0.0:1788571832.833292:0:2384:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016a00/0xef30c340bfb3d3d4 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.833298:0:2384:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016a00/0xef30c340bfb3d3d4 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.833302:0:2384:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016a00/0xef30c340bfb3d3d4 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.833308:0:2384:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1788571832.833311:0:2384:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016a00/0xef30c340bfb3d3d4 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.833317:0:2384:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016a00/0xef30c340bfb3d3d4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.833329:0:2384:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017e00/0xef30c340bfb3d3db lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.833335:0:2384:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017e00/0xef30c340bfb3d3db lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.833345:0:2384:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017e00/0xef30c340bfb3d3db lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.833351:0:2384:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017e00/0xef30c340bfb3d3db lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.833356:0:2384:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017e00/0xef30c340bfb3d3db lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.833373:0:2384:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada53017e00 already unused, calling callback (ffffffffc1966630) 00010000:00010000:0.0:1788571832.833376:0:2384:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017e00/0xef30c340bfb3d3db lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 3 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.833383:0:2384:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017e00/0xef30c340bfb3d3db lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 3 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.833389:0:2384:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017e00/0xef30c340bfb3d3db lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 3 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.833395:0:2384:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1788571832.833398:0:2384:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017e00/0xef30c340bfb3d3db lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.833403:0:2384:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017e00/0xef30c340bfb3d3db lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571832.833426:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613592448:192.168.202.113@tcp:63:ll_sa.0 00010000:00010000:0.0:1788571832.838520:0:2384:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 000000009dd3094c/0xef30c340bfb3d3f0 lrc: 6/2,0 mode: --/PR res: [0x280000401:0x606:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 79358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.838551:0:2384:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 000000009dd3094c/0xef30c340bfb3d3f0 lrc: 6/2,0 mode: --/PR res: [0x280000401:0x606:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f72fe expref: -99 pid: 79358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.838564:0:2384:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 000000009dd3094c/0xef30c340bfb3d3f0 lrc: 6/2,0 mode: PR/PR res: [0x280000401:0x606:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f72fe expref: -99 pid: 79358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.838590:0:2384:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 000000009dd3094c/0xef30c340bfb3d3f0 lrc: 6/2,0 mode: PR/PR res: [0x280000401:0x606:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f72fe expref: -99 pid: 79358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1788571832.838606:0:2384:0:(osc_lock.c:187:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 000000009dd3094c/0xef30c340bfb3d3f0 lrc: 5/2,0 mode: PR/PR res: [0x280000401:0x606:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f72fe expref: -99 pid: 79358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.838622:0:2384:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 000000009dd3094c/0xef30c340bfb3d3f0 lrc: 5/2,0 mode: PR/PR res: [0x280000401:0x606:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f72fe expref: -99 pid: 79358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.838630:0:2384:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 000000009dd3094c/0xef30c340bfb3d3f0 lrc: 4/1,0 mode: PR/PR res: [0x280000401:0x606:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f72fe expref: -99 pid: 79358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.838639:0:2384:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 000000009dd3094c/0xef30c340bfb3d3f0 lrc: 4/1,0 mode: PR/PR res: [0x280000401:0x606:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f72fe expref: -99 pid: 79358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.838651:0:2384:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 000000009dd3094c/0xef30c340bfb3d3f0 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x606:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f72fe expref: -99 pid: 79358 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:0.0:1788571832.838680:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613592832:192.168.202.113@tcp:101:ll_sa.0 00010000:00010000:0.0:1788571832.843404:0:2384:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 00000000c4ab1af0/0xef30c340bfb3d3e2 lrc: 6/2,0 mode: --/PR res: [0x2c0000401:0x605:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 79358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.843444:0:2384:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 00000000c4ab1af0/0xef30c340bfb3d3e2 lrc: 6/2,0 mode: --/PR res: [0x2c0000401:0x605:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7305 expref: -99 pid: 79358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.843462:0:2384:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 00000000c4ab1af0/0xef30c340bfb3d3e2 lrc: 6/2,0 mode: PR/PR res: [0x2c0000401:0x605:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7305 expref: -99 pid: 79358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.843477:0:2384:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 00000000c4ab1af0/0xef30c340bfb3d3e2 lrc: 6/2,0 mode: PR/PR res: [0x2c0000401:0x605:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7305 expref: -99 pid: 79358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1788571832.843498:0:2384:0:(osc_lock.c:187:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 00000000c4ab1af0/0xef30c340bfb3d3e2 lrc: 5/2,0 mode: PR/PR res: [0x2c0000401:0x605:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f7305 expref: -99 pid: 79358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.843515:0:2384:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 00000000c4ab1af0/0xef30c340bfb3d3e2 lrc: 5/2,0 mode: PR/PR res: [0x2c0000401:0x605:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f7305 expref: -99 pid: 79358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.843524:0:2384:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 00000000c4ab1af0/0xef30c340bfb3d3e2 lrc: 4/1,0 mode: PR/PR res: [0x2c0000401:0x605:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f7305 expref: -99 pid: 79358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.843533:0:2384:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 00000000c4ab1af0/0xef30c340bfb3d3e2 lrc: 4/1,0 mode: PR/PR res: [0x2c0000401:0x605:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f7305 expref: -99 pid: 79358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.843546:0:2384:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 00000000c4ab1af0/0xef30c340bfb3d3e2 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x605:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f7305 expref: -99 pid: 79358 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:0.0:1788571832.843580:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613592576:192.168.202.113@tcp:101:ll_sa.0 00000080:00200000:2.0:1788571832.844237:0:79357:0:(file.c:1775:ll_merge_attr_nolock()) [0x200016b60:0x7:0x0] updating i_size 0 i_blocks 0 00000020:00010000:2.0:1788571832.844245:0:79357:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9ada60bc39d0 (ffff9ada60b0d248/0) at cl_lock_release():245 00000020:00010000:2.0:1788571832.844249:0:79357:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9ada60bc39d0 (ffff9ada60b0d248/0) at cl_lock_cancel():142 00000020:00010000:2.0:1788571832.844255:0:79357:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9ada48b9ffb0 (ffff9ada60b0d098/1) at cl_lock_cancel():142 00000020:00010000:2.0:1788571832.844260:0:79357:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9ada60bc39d0 (ffff9ada60b0d248/0) at cl_lock_fini():72 00000020:00010000:2.0:1788571832.844263:0:79357:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9ada48b9ffb0 (ffff9ada60b0d248/1) at cl_lock_fini():72 00000080:00200000:2.0:1788571832.844280:0:79357:0:(vvp_io.c:304:vvp_io_fini()) [0x200016b60:0x7:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1788571832.844337:0:79357:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571832.844378:0:79357:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571832.844400:0:79357:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d3b8 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f72db expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571832.844415:0:79357:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d3b8 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f72db expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571832.844424:0:79357:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571832.844433:0:79357:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571832.844437:0:79357:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d3b8 00000080:00010000:2.0:1788571832.844444:0:79357:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d3b8 from it ffffab0ec1d37c10 00010000:00010000:2.0:1788571832.844449:0:79357:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d3b8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f72db expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571832.844457:0:79357:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d3b8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f72db expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571832.844469:0:79357:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:2.0:1788571832.844482:0:79357:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f33b.replay-single--6, flags=0 00000080:00200000:2.0:1788571832.844509:0:79357:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571832.844517:0:79357:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571832.844523:0:79357:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d3b8 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f72db expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571832.844531:0:79357:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d3b8 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f72db expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571832.844538:0:79357:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571832.844542:0:79357:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571832.844546:0:79357:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d3b8 00000080:00010000:2.0:1788571832.844549:0:79357:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d3b8 from it ffffab0ec1d37d68 00010000:00010000:2.0:1788571832.844552:0:79357:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d3b8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f72db expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571832.844559:0:79357:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d3b8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f72db expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1788571832.844750:0:79357:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:2.0:1788571832.844759:0:79357:0:(namei.c:2351:ll_unlink()) VFS Op:name=f33b.replay-single--6, dir=[0x200000007:0x1:0x0](ffff9ada58125b88) 00010000:00010000:2.0:1788571832.844795:0:79357:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d3b8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b82f72db expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1788571832.844827:0:79357:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f000/0xef30c340bfb3d333 lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0xdb7edb52b82f710d expref: -99 pid: 79356 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571832.844839:0:79357:0:(vvp_object.c:117:vvp_conf_set()) [0x200016b60:0x7:0x0]: losing layout lock 00010000:00010000:2.0:1788571832.844922:0:79357:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f000/0xef30c340bfb3d333 lrc: 1/0,0 mode: --/PR res: [0x200016b60:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0xdb7edb52b82f710d expref: -99 pid: 79356 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571832.844931:0:79357:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d3b8 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f72db expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1788571832.844938:0:79357:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:2.0:1788571832.844942:0:79357:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5163f000/0xef30c340bfb3d333 lrc: 0/0,0 mode: --/PR res: [0x200016b60:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0xdb7edb52b82f710d expref: -99 pid: 79356 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571832.844968:0:79357:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d3b8 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f72db expref: -99 pid: 79357 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:2.0:1788571832.845160:0:79357:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1788571832.845174:0:79357:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada60a34e00 x1875449613593216/t0(0) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788571832.845205:0:79357:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a34e00 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:79357:1875449613593216:192.168.202.113@tcp:36:rm.0 00000100:00100000:2.0:1788571832.845371:0:79357:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6059b780 going to sleep for 11 seconds 00000100:00100000:0.0:1788571832.862512:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1788571832.862547:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613593728:192.168.202.113@tcp:63:ll_sa.0 00010000:00010000:0.0:1788571832.871632:0:2384:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016600/0xef30c340bfb3d405 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.871657:0:2384:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016600/0xef30c340bfb3d405 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.871666:0:2384:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016600/0xef30c340bfb3d405 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.871675:0:2384:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016600/0xef30c340bfb3d405 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.871686:0:2384:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016600/0xef30c340bfb3d405 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.871693:0:2384:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada53016600 already unused, calling callback (ffffffffc1966630) 00010000:00010000:0.0:1788571832.871699:0:2384:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016600/0xef30c340bfb3d405 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.871707:0:2384:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016600/0xef30c340bfb3d405 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.871719:0:2384:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016600/0xef30c340bfb3d405 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.871728:0:2384:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1788571832.871744:0:2384:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016600/0xef30c340bfb3d405 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.871752:0:2384:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016600/0xef30c340bfb3d405 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.871778:0:2384:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017400/0xef30c340bfb3d40c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 10 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.871784:0:2384:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017400/0xef30c340bfb3d40c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 10 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.871790:0:2384:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017400/0xef30c340bfb3d40c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 10 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.871795:0:2384:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017400/0xef30c340bfb3d40c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 10 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.871801:0:2384:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017400/0xef30c340bfb3d40c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 10 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.871805:0:2384:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada53017400 already unused, calling callback (ffffffffc1966630) 00010000:00010000:0.0:1788571832.871808:0:2384:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017400/0xef30c340bfb3d40c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 10 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.871813:0:2384:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017400/0xef30c340bfb3d40c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 10 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.871823:0:2384:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017400/0xef30c340bfb3d40c lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 10 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.871828:0:2384:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1788571832.871831:0:2384:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017400/0xef30c340bfb3d40c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 10 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.871838:0:2384:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017400/0xef30c340bfb3d40c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 10 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.871851:0:2384:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d413 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 9 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872012:0:2384:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d413 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 9 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872020:0:2384:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d413 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 9 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872026:0:2384:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d413 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 9 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872033:0:2384:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d413 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 9 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872038:0:2384:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada53017600 already unused, calling callback (ffffffffc1966630) 00010000:00010000:0.0:1788571832.872172:0:2384:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d413 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 9 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872182:0:2384:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d413 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 9 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872193:0:2384:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d413 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 9 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872199:0:2384:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1788571832.872203:0:2384:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d413 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 9 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872209:0:2384:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d413 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 9 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872225:0:2384:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016800/0xef30c340bfb3d41a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872231:0:2384:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016800/0xef30c340bfb3d41a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872238:0:2384:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016800/0xef30c340bfb3d41a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872243:0:2384:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016800/0xef30c340bfb3d41a lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872250:0:2384:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016800/0xef30c340bfb3d41a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872255:0:2384:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada53016800 already unused, calling callback (ffffffffc1966630) 00010000:00010000:0.0:1788571832.872258:0:2384:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016800/0xef30c340bfb3d41a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 8 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872263:0:2384:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016800/0xef30c340bfb3d41a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 8 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872272:0:2384:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016800/0xef30c340bfb3d41a lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 8 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872278:0:2384:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1788571832.872281:0:2384:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016800/0xef30c340bfb3d41a lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 8 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872287:0:2384:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016800/0xef30c340bfb3d41a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 8 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872301:0:2384:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016200/0xef30c340bfb3d421 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872307:0:2384:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016200/0xef30c340bfb3d421 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872314:0:2384:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016200/0xef30c340bfb3d421 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872319:0:2384:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016200/0xef30c340bfb3d421 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872325:0:2384:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016200/0xef30c340bfb3d421 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872331:0:2384:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada53016200 already unused, calling callback (ffffffffc1966630) 00010000:00010000:0.0:1788571832.872334:0:2384:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016200/0xef30c340bfb3d421 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 7 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872341:0:2384:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016200/0xef30c340bfb3d421 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 7 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872352:0:2384:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016200/0xef30c340bfb3d421 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 7 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872371:0:2384:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1788571832.872375:0:2384:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016200/0xef30c340bfb3d421 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 7 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872382:0:2384:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016200/0xef30c340bfb3d421 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 7 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872396:0:2384:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016e00/0xef30c340bfb3d428 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872402:0:2384:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016e00/0xef30c340bfb3d428 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872410:0:2384:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016e00/0xef30c340bfb3d428 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872416:0:2384:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016e00/0xef30c340bfb3d428 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872423:0:2384:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016e00/0xef30c340bfb3d428 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872428:0:2384:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada53016e00 already unused, calling callback (ffffffffc1966630) 00010000:00010000:0.0:1788571832.872430:0:2384:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016e00/0xef30c340bfb3d428 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 6 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872436:0:2384:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016e00/0xef30c340bfb3d428 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 6 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872445:0:2384:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016e00/0xef30c340bfb3d428 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 6 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872450:0:2384:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1788571832.872454:0:2384:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016e00/0xef30c340bfb3d428 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 6 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872460:0:2384:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016e00/0xef30c340bfb3d428 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 6 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872473:0:2384:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016400/0xef30c340bfb3d42f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872478:0:2384:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016400/0xef30c340bfb3d42f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872485:0:2384:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016400/0xef30c340bfb3d42f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872491:0:2384:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016400/0xef30c340bfb3d42f lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872497:0:2384:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016400/0xef30c340bfb3d42f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872502:0:2384:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada53016400 already unused, calling callback (ffffffffc1966630) 00010000:00010000:0.0:1788571832.872505:0:2384:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016400/0xef30c340bfb3d42f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 5 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872511:0:2384:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016400/0xef30c340bfb3d42f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 5 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872521:0:2384:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016400/0xef30c340bfb3d42f lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 5 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872527:0:2384:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1788571832.872530:0:2384:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016400/0xef30c340bfb3d42f lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 5 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872535:0:2384:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016400/0xef30c340bfb3d42f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 5 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872551:0:2384:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017e00/0xef30c340bfb3d436 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872557:0:2384:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017e00/0xef30c340bfb3d436 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872564:0:2384:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017e00/0xef30c340bfb3d436 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872683:0:2384:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017e00/0xef30c340bfb3d436 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872691:0:2384:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017e00/0xef30c340bfb3d436 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872697:0:2384:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada53017e00 already unused, calling callback (ffffffffc1966630) 00010000:00010000:0.0:1788571832.872700:0:2384:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017e00/0xef30c340bfb3d436 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872706:0:2384:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017e00/0xef30c340bfb3d436 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872716:0:2384:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017e00/0xef30c340bfb3d436 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872722:0:2384:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1788571832.872726:0:2384:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017e00/0xef30c340bfb3d436 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872733:0:2384:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017e00/0xef30c340bfb3d436 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872757:0:2384:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016a00/0xef30c340bfb3d43d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872763:0:2384:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016a00/0xef30c340bfb3d43d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872769:0:2384:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016a00/0xef30c340bfb3d43d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872775:0:2384:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016a00/0xef30c340bfb3d43d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872782:0:2384:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016a00/0xef30c340bfb3d43d lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872788:0:2384:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada53016a00 already unused, calling callback (ffffffffc1966630) 00010000:00010000:0.0:1788571832.872791:0:2384:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016a00/0xef30c340bfb3d43d lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 3 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872797:0:2384:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016a00/0xef30c340bfb3d43d lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 3 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872806:0:2384:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016a00/0xef30c340bfb3d43d lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 3 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872811:0:2384:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1788571832.872815:0:2384:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016a00/0xef30c340bfb3d43d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571832.872821:0:2384:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016a00/0xef30c340bfb3d43d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571832.872847:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613593728:192.168.202.113@tcp:63:ll_sa.0 00010000:00010000:2.0:1788571832.900747:0:79358:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172cc00/0xef30c340bfb3d4c9 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT flags: 0x10000000000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.900756:0:79358:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172cc00/0xef30c340bfb3d4c9 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 9 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.900784:0:79358:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c800/0xef30c340bfb3d4d0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 10 type: IBT flags: 0x10000000000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.900791:0:79358:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c800/0xef30c340bfb3d4d0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 10 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.901712:0:79358:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ca00/0xef30c340bfb3d4d7 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT flags: 0x10000000000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.901721:0:79358:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ca00/0xef30c340bfb3d4d7 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.901754:0:79358:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d800/0xef30c340bfb3d4de lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT flags: 0x10000000000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.901762:0:79358:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d800/0xef30c340bfb3d4de lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 9 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.901786:0:79358:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c400/0xef30c340bfb3d4e5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 10 type: IBT flags: 0x10000000000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.901792:0:79358:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c400/0xef30c340bfb3d4e5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 10 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.901809:0:79358:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d4ec lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 11 type: IBT flags: 0x10000000000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.901816:0:79358:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d4ec lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.901834:0:79358:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c200/0xef30c340bfb3d4f3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 12 type: IBT flags: 0x10000000000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.901840:0:79358:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c200/0xef30c340bfb3d4f3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.902128:0:79358:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172da00/0xef30c340bfb3d4fa lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x10000000000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.902136:0:79358:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172da00/0xef30c340bfb3d4fa lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 13 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.902155:0:79358:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c000/0xef30c340bfb3d501 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 14 type: IBT flags: 0x10000000000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.902161:0:79358:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c000/0xef30c340bfb3d501 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.902176:0:79358:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d600/0xef30c340bfb3d508 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 15 type: IBT flags: 0x10000000000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.902183:0:79358:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d600/0xef30c340bfb3d508 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.902201:0:79358:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d000/0xef30c340bfb3d50f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 16 type: IBT flags: 0x10000000000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.902206:0:79358:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d000/0xef30c340bfb3d50f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.902219:0:79358:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c600/0xef30c340bfb3d516 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 17 type: IBT flags: 0x10000000000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.902225:0:79358:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c600/0xef30c340bfb3d516 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.902238:0:79358:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172dc00/0xef30c340bfb3d51d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 18 type: IBT flags: 0x10000000000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.902245:0:79358:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172dc00/0xef30c340bfb3d51d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.902259:0:79358:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d524 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 19 type: IBT flags: 0x10000000000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.902265:0:79358:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d524 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 19 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.902279:0:79358:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17e00/0xef30c340bfb3d52b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 20 type: IBT flags: 0x10000000000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.902286:0:79358:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17e00/0xef30c340bfb3d52b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 20 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.902302:0:79358:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17400/0xef30c340bfb3d532 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 21 type: IBT flags: 0x10000000000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.902309:0:79358:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17400/0xef30c340bfb3d532 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 21 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.902323:0:79358:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16c00/0xef30c340bfb3d539 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 22 type: IBT flags: 0x10000000000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.902329:0:79358:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16c00/0xef30c340bfb3d539 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 22 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.902344:0:79358:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17a00/0xef30c340bfb3d540 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 23 type: IBT flags: 0x10000000000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.902350:0:79358:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17a00/0xef30c340bfb3d540 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 23 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.903479:0:79358:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17800/0xef30c340bfb3d547 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 21 type: IBT flags: 0x10000000000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.903487:0:79358:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17800/0xef30c340bfb3d547 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 21 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.903509:0:79358:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16000/0xef30c340bfb3d54e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 22 type: IBT flags: 0x10000000000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.903515:0:79358:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16000/0xef30c340bfb3d54e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 22 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.903539:0:79358:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16e00/0xef30c340bfb3d555 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 23 type: IBT flags: 0x10000000000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.903546:0:79358:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16e00/0xef30c340bfb3d555 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 23 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.903567:0:79358:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17200/0xef30c340bfb3d55c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 24 type: IBT flags: 0x10000000000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.903641:0:79358:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17200/0xef30c340bfb3d55c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 24 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.903667:0:79358:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16600/0xef30c340bfb3d563 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 25 type: IBT flags: 0x10000000000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.903674:0:79358:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16600/0xef30c340bfb3d563 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 25 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.903693:0:79358:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17000/0xef30c340bfb3d56a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 26 type: IBT flags: 0x10000000000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.903699:0:79358:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17000/0xef30c340bfb3d56a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 26 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.903717:0:79358:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16a00/0xef30c340bfb3d571 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 27 type: IBT flags: 0x10000000000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.903724:0:79358:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16a00/0xef30c340bfb3d571 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 27 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.903750:0:79358:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17c00/0xef30c340bfb3d578 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 28 type: IBT flags: 0x10000000000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.903758:0:79358:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17c00/0xef30c340bfb3d578 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 28 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.903777:0:79358:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17600/0xef30c340bfb3d57f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 29 type: IBT flags: 0x10000000000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.903783:0:79358:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17600/0xef30c340bfb3d57f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 29 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.903802:0:79358:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16400/0xef30c340bfb3d586 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 30 type: IBT flags: 0x10000000000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.903808:0:79358:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16400/0xef30c340bfb3d586 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 30 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.903826:0:79358:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16800/0xef30c340bfb3d58d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 31 type: IBT flags: 0x10000000000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.903833:0:79358:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16800/0xef30c340bfb3d58d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 31 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.903850:0:79358:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16200/0xef30c340bfb3d594 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 32 type: IBT flags: 0x10000000000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.904137:0:79358:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16200/0xef30c340bfb3d594 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 33 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.904830:0:79358:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0c00/0xef30c340bfb3d59b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 32 type: IBT flags: 0x10000000000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.904839:0:79358:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0c00/0xef30c340bfb3d59b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 32 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.904913:0:79358:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1200/0xef30c340bfb3d5a2 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 33 type: IBT flags: 0x10000000000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.904922:0:79358:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1200/0xef30c340bfb3d5a2 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 33 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.904944:0:79358:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1400/0xef30c340bfb3d5a9 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 34 type: IBT flags: 0x10000000000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.904951:0:79358:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1400/0xef30c340bfb3d5a9 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 34 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571832.905036:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a36a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613594752:192.168.202.113@tcp:63:ll_sa.0 00010000:00010000:2.0:1788571832.920750:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172cc00/0xef30c340bfb3d4c9 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 34 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.920777:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172cc00/0xef30c340bfb3d4c9 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 34 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.920791:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172cc00/0xef30c340bfb3d4c9 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 34 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.920800:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172cc00/0xef30c340bfb3d4c9 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 34 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.920831:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172cc00/0xef30c340bfb3d4c9 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 34 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.920840:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada5172cc00 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571832.920846:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172cc00/0xef30c340bfb3d4c9 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 34 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.920889:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172cc00/0xef30c340bfb3d4c9 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 34 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.920900:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172cc00/0xef30c340bfb3d4c9 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 34 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.920915:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571832.920926:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172cc00/0xef30c340bfb3d4c9 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 34 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.920939:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172cc00/0xef30c340bfb3d4c9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 34 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.920977:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c800/0xef30c340bfb3d4d0 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 33 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.920989:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c800/0xef30c340bfb3d4d0 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 33 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.920995:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c800/0xef30c340bfb3d4d0 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 33 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921001:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c800/0xef30c340bfb3d4d0 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 33 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921012:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c800/0xef30c340bfb3d4d0 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 33 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921018:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada5172c800 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571832.921021:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c800/0xef30c340bfb3d4d0 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 33 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921027:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c800/0xef30c340bfb3d4d0 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 33 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921043:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c800/0xef30c340bfb3d4d0 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 33 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921048:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571832.921052:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c800/0xef30c340bfb3d4d0 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 33 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921058:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c800/0xef30c340bfb3d4d0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 33 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921077:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ca00/0xef30c340bfb3d4d7 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 32 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921084:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ca00/0xef30c340bfb3d4d7 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 32 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921090:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ca00/0xef30c340bfb3d4d7 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 32 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921096:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ca00/0xef30c340bfb3d4d7 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 32 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921103:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ca00/0xef30c340bfb3d4d7 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 32 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921108:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada5172ca00 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571832.921111:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ca00/0xef30c340bfb3d4d7 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 32 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921117:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ca00/0xef30c340bfb3d4d7 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 32 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921127:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ca00/0xef30c340bfb3d4d7 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 32 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921133:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571832.921136:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ca00/0xef30c340bfb3d4d7 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 32 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921142:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ca00/0xef30c340bfb3d4d7 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 32 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921156:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d800/0xef30c340bfb3d4de lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 31 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921163:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d800/0xef30c340bfb3d4de lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 31 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921169:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d800/0xef30c340bfb3d4de lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 31 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921175:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d800/0xef30c340bfb3d4de lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 31 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921181:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d800/0xef30c340bfb3d4de lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 31 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921187:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada5172d800 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571832.921190:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d800/0xef30c340bfb3d4de lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 31 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921196:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d800/0xef30c340bfb3d4de lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 31 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921214:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d800/0xef30c340bfb3d4de lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 31 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921221:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571832.921224:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d800/0xef30c340bfb3d4de lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 31 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921236:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d800/0xef30c340bfb3d4de lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 31 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921249:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c400/0xef30c340bfb3d4e5 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 30 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921261:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c400/0xef30c340bfb3d4e5 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 30 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921267:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c400/0xef30c340bfb3d4e5 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 30 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921272:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c400/0xef30c340bfb3d4e5 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 30 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921283:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c400/0xef30c340bfb3d4e5 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 30 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921289:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada5172c400 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571832.921292:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c400/0xef30c340bfb3d4e5 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 30 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921298:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c400/0xef30c340bfb3d4e5 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 30 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921306:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c400/0xef30c340bfb3d4e5 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 30 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921311:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571832.921315:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c400/0xef30c340bfb3d4e5 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 30 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921322:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c400/0xef30c340bfb3d4e5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 30 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921336:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d4ec lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 29 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921342:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d4ec lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 29 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921348:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d4ec lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 29 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921354:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d4ec lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 29 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921361:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d4ec lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 29 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921366:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada5172ce00 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571832.921369:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d4ec lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 29 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921375:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d4ec lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 29 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921384:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d4ec lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 29 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921390:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571832.921393:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d4ec lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 29 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921399:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3d4ec lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 29 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921414:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c200/0xef30c340bfb3d4f3 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 28 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921420:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c200/0xef30c340bfb3d4f3 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 28 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921426:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c200/0xef30c340bfb3d4f3 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 28 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921432:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c200/0xef30c340bfb3d4f3 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 28 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921438:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c200/0xef30c340bfb3d4f3 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 28 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921444:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada5172c200 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571832.921446:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c200/0xef30c340bfb3d4f3 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 28 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921452:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c200/0xef30c340bfb3d4f3 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 28 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921462:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c200/0xef30c340bfb3d4f3 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 28 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921468:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571832.921471:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c200/0xef30c340bfb3d4f3 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 28 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921478:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c200/0xef30c340bfb3d4f3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 28 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921492:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172da00/0xef30c340bfb3d4fa lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 27 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921498:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172da00/0xef30c340bfb3d4fa lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 27 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921503:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172da00/0xef30c340bfb3d4fa lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 27 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921510:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172da00/0xef30c340bfb3d4fa lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 27 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921516:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172da00/0xef30c340bfb3d4fa lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 27 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921522:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada5172da00 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571832.921525:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172da00/0xef30c340bfb3d4fa lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 27 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921530:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172da00/0xef30c340bfb3d4fa lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 27 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921539:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172da00/0xef30c340bfb3d4fa lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 27 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921544:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571832.921547:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172da00/0xef30c340bfb3d4fa lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 27 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921554:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172da00/0xef30c340bfb3d4fa lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 27 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921568:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c000/0xef30c340bfb3d501 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 26 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921575:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c000/0xef30c340bfb3d501 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 26 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921580:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c000/0xef30c340bfb3d501 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 26 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921587:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c000/0xef30c340bfb3d501 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 26 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921593:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c000/0xef30c340bfb3d501 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 26 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921598:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada5172c000 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571832.921601:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c000/0xef30c340bfb3d501 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 26 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921607:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c000/0xef30c340bfb3d501 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 26 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921616:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c000/0xef30c340bfb3d501 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 26 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921621:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571832.921625:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c000/0xef30c340bfb3d501 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 26 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921632:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c000/0xef30c340bfb3d501 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 26 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921646:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d600/0xef30c340bfb3d508 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 25 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921652:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d600/0xef30c340bfb3d508 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 25 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921658:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d600/0xef30c340bfb3d508 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 25 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921664:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d600/0xef30c340bfb3d508 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 25 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921671:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d600/0xef30c340bfb3d508 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 25 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921677:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada5172d600 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571832.921680:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d600/0xef30c340bfb3d508 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 25 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921687:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d600/0xef30c340bfb3d508 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 25 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921695:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d600/0xef30c340bfb3d508 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 25 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921700:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571832.921703:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d600/0xef30c340bfb3d508 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 25 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921709:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d600/0xef30c340bfb3d508 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 25 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921721:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d000/0xef30c340bfb3d50f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 24 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921726:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d000/0xef30c340bfb3d50f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 24 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.921732:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d000/0xef30c340bfb3d50f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 24 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.925555:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d000/0xef30c340bfb3d50f lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 24 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.925640:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d000/0xef30c340bfb3d50f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 24 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.925649:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada5172d000 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571832.925656:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d000/0xef30c340bfb3d50f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 24 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.925666:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d000/0xef30c340bfb3d50f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 24 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.925685:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d000/0xef30c340bfb3d50f lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 24 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.925695:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571832.925705:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d000/0xef30c340bfb3d50f lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 24 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.925714:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d000/0xef30c340bfb3d50f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 24 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.925758:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c600/0xef30c340bfb3d516 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 23 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.925766:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c600/0xef30c340bfb3d516 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 23 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.925774:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c600/0xef30c340bfb3d516 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 23 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.925780:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c600/0xef30c340bfb3d516 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 23 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.925787:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c600/0xef30c340bfb3d516 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 23 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.925792:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada5172c600 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571832.925795:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c600/0xef30c340bfb3d516 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 23 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.926175:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c600/0xef30c340bfb3d516 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 23 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.926188:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c600/0xef30c340bfb3d516 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 23 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.926195:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571832.926200:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c600/0xef30c340bfb3d516 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 23 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.926207:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c600/0xef30c340bfb3d516 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 23 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.926228:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172dc00/0xef30c340bfb3d51d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 22 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.926234:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172dc00/0xef30c340bfb3d51d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 22 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.926240:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172dc00/0xef30c340bfb3d51d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 22 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.926245:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172dc00/0xef30c340bfb3d51d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 22 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.926251:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172dc00/0xef30c340bfb3d51d lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 22 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.926256:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada5172dc00 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571832.926258:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172dc00/0xef30c340bfb3d51d lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 22 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.926264:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172dc00/0xef30c340bfb3d51d lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 22 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.926273:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172dc00/0xef30c340bfb3d51d lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 22 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.926277:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571832.926281:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172dc00/0xef30c340bfb3d51d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 22 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.926286:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172dc00/0xef30c340bfb3d51d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 22 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.926299:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d524 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 21 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.926306:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d524 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 21 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.926312:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d524 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 21 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.926318:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d524 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 21 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.926325:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d524 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 21 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.926330:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada5172de00 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571832.926333:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d524 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 21 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.926339:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d524 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 21 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.926349:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d524 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 21 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.926354:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571832.927014:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d524 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 21 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.927027:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172de00/0xef30c340bfb3d524 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 21 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.927473:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17e00/0xef30c340bfb3d52b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 20 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.927484:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17e00/0xef30c340bfb3d52b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 20 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.927502:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17e00/0xef30c340bfb3d52b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 20 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.927510:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17e00/0xef30c340bfb3d52b lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 20 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.927517:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17e00/0xef30c340bfb3d52b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 20 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.927523:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada60a17e00 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571832.927527:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17e00/0xef30c340bfb3d52b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 20 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.927535:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17e00/0xef30c340bfb3d52b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 20 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.927547:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17e00/0xef30c340bfb3d52b lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 20 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.927554:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571832.927559:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17e00/0xef30c340bfb3d52b lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 20 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.927566:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17e00/0xef30c340bfb3d52b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 20 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.927591:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17400/0xef30c340bfb3d532 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 19 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.927597:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17400/0xef30c340bfb3d532 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 19 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.927603:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17400/0xef30c340bfb3d532 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 19 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.927609:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17400/0xef30c340bfb3d532 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 19 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.927615:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17400/0xef30c340bfb3d532 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 19 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.927621:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada60a17400 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571832.927632:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17400/0xef30c340bfb3d532 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 19 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.927639:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17400/0xef30c340bfb3d532 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 19 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.927649:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17400/0xef30c340bfb3d532 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 19 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.927659:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571832.927663:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17400/0xef30c340bfb3d532 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 19 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.927670:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17400/0xef30c340bfb3d532 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 19 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.927692:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16c00/0xef30c340bfb3d539 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.927698:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16c00/0xef30c340bfb3d539 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.927704:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16c00/0xef30c340bfb3d539 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.927715:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16c00/0xef30c340bfb3d539 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.927722:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16c00/0xef30c340bfb3d539 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 18 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.927729:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada60a16c00 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571832.927733:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16c00/0xef30c340bfb3d539 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.927752:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16c00/0xef30c340bfb3d539 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.927762:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16c00/0xef30c340bfb3d539 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.927769:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571832.927773:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16c00/0xef30c340bfb3d539 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.927786:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16c00/0xef30c340bfb3d539 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 18 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.927924:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17a00/0xef30c340bfb3d540 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.927932:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17a00/0xef30c340bfb3d540 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.927939:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17a00/0xef30c340bfb3d540 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.927945:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17a00/0xef30c340bfb3d540 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.927952:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17a00/0xef30c340bfb3d540 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 17 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.927957:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada60a17a00 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571832.927960:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17a00/0xef30c340bfb3d540 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.927966:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17a00/0xef30c340bfb3d540 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.927976:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17a00/0xef30c340bfb3d540 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.927982:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571832.927985:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17a00/0xef30c340bfb3d540 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.927992:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17a00/0xef30c340bfb3d540 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 17 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928007:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17800/0xef30c340bfb3d547 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928012:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17800/0xef30c340bfb3d547 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928027:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17800/0xef30c340bfb3d547 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928033:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17800/0xef30c340bfb3d547 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928040:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17800/0xef30c340bfb3d547 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 16 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928246:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada60a17800 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571832.928250:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17800/0xef30c340bfb3d547 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928256:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17800/0xef30c340bfb3d547 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928274:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17800/0xef30c340bfb3d547 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928280:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571832.928284:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17800/0xef30c340bfb3d547 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928296:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17800/0xef30c340bfb3d547 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 16 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928312:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16000/0xef30c340bfb3d54e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928324:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16000/0xef30c340bfb3d54e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928331:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16000/0xef30c340bfb3d54e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928339:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16000/0xef30c340bfb3d54e lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928419:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16000/0xef30c340bfb3d54e lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 15 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928428:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada60a16000 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571832.928432:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16000/0xef30c340bfb3d54e lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928438:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16000/0xef30c340bfb3d54e lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928448:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16000/0xef30c340bfb3d54e lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928454:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571832.928458:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16000/0xef30c340bfb3d54e lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928471:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16000/0xef30c340bfb3d54e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 15 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928490:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16e00/0xef30c340bfb3d555 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928502:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16e00/0xef30c340bfb3d555 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928508:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16e00/0xef30c340bfb3d555 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928513:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16e00/0xef30c340bfb3d555 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928519:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16e00/0xef30c340bfb3d555 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 14 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928525:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada60a16e00 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571832.928527:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16e00/0xef30c340bfb3d555 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 14 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928534:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16e00/0xef30c340bfb3d555 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 14 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928544:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16e00/0xef30c340bfb3d555 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 14 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928550:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571832.928554:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16e00/0xef30c340bfb3d555 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 14 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928560:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16e00/0xef30c340bfb3d555 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 14 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928683:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17200/0xef30c340bfb3d55c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 13 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928697:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17200/0xef30c340bfb3d55c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 13 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928703:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17200/0xef30c340bfb3d55c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 13 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928709:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17200/0xef30c340bfb3d55c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 13 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928721:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17200/0xef30c340bfb3d55c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 13 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928727:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada60a17200 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571832.928729:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17200/0xef30c340bfb3d55c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 13 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928746:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17200/0xef30c340bfb3d55c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 13 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928756:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17200/0xef30c340bfb3d55c lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 13 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928762:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571832.928765:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17200/0xef30c340bfb3d55c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 13 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928771:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17200/0xef30c340bfb3d55c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 13 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928792:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16600/0xef30c340bfb3d563 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928916:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16600/0xef30c340bfb3d563 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928924:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16600/0xef30c340bfb3d563 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928930:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16600/0xef30c340bfb3d563 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928937:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16600/0xef30c340bfb3d563 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 12 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928943:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada60a16600 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571832.928946:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16600/0xef30c340bfb3d563 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 12 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928952:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16600/0xef30c340bfb3d563 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 12 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928971:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16600/0xef30c340bfb3d563 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 12 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928976:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571832.928980:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16600/0xef30c340bfb3d563 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 12 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.928987:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16600/0xef30c340bfb3d563 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 12 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929010:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17000/0xef30c340bfb3d56a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929021:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17000/0xef30c340bfb3d56a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929028:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17000/0xef30c340bfb3d56a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929033:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17000/0xef30c340bfb3d56a lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929190:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17000/0xef30c340bfb3d56a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 11 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929205:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada60a17000 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571832.929209:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17000/0xef30c340bfb3d56a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929215:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17000/0xef30c340bfb3d56a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929230:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17000/0xef30c340bfb3d56a lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929237:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571832.929241:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17000/0xef30c340bfb3d56a lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929247:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17000/0xef30c340bfb3d56a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 11 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929269:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16a00/0xef30c340bfb3d571 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 10 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929276:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16a00/0xef30c340bfb3d571 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 10 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929282:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16a00/0xef30c340bfb3d571 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 10 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929293:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16a00/0xef30c340bfb3d571 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 10 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929300:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16a00/0xef30c340bfb3d571 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 10 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929305:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada60a16a00 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571832.929308:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16a00/0xef30c340bfb3d571 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 10 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929321:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16a00/0xef30c340bfb3d571 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 10 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929330:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16a00/0xef30c340bfb3d571 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 10 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929335:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571832.929339:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16a00/0xef30c340bfb3d571 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 10 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929346:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16a00/0xef30c340bfb3d571 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 10 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929372:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17c00/0xef30c340bfb3d578 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 9 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929379:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17c00/0xef30c340bfb3d578 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 9 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929385:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17c00/0xef30c340bfb3d578 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 9 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929391:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17c00/0xef30c340bfb3d578 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 9 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929398:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17c00/0xef30c340bfb3d578 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 9 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929403:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada60a17c00 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571832.929406:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17c00/0xef30c340bfb3d578 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 9 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929412:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17c00/0xef30c340bfb3d578 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 9 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929422:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17c00/0xef30c340bfb3d578 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 9 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929427:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571832.929431:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17c00/0xef30c340bfb3d578 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 9 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929438:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17c00/0xef30c340bfb3d578 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 9 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929451:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17600/0xef30c340bfb3d57f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929457:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17600/0xef30c340bfb3d57f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929462:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17600/0xef30c340bfb3d57f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929468:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17600/0xef30c340bfb3d57f lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929475:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17600/0xef30c340bfb3d57f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 8 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929480:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada60a17600 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571832.929483:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17600/0xef30c340bfb3d57f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 8 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929489:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17600/0xef30c340bfb3d57f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 8 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929500:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17600/0xef30c340bfb3d57f lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 8 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929505:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571832.929509:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17600/0xef30c340bfb3d57f lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 8 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929516:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17600/0xef30c340bfb3d57f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 8 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929529:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16400/0xef30c340bfb3d586 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929535:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16400/0xef30c340bfb3d586 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929541:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16400/0xef30c340bfb3d586 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929547:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16400/0xef30c340bfb3d586 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929554:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16400/0xef30c340bfb3d586 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 7 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929559:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada60a16400 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571832.929562:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16400/0xef30c340bfb3d586 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 7 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929679:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16400/0xef30c340bfb3d586 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 7 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929693:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16400/0xef30c340bfb3d586 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 7 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929699:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571832.929703:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16400/0xef30c340bfb3d586 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 7 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929710:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16400/0xef30c340bfb3d586 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 7 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929725:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16800/0xef30c340bfb3d58d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929730:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16800/0xef30c340bfb3d58d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929746:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16800/0xef30c340bfb3d58d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929751:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16800/0xef30c340bfb3d58d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929757:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16800/0xef30c340bfb3d58d lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 6 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929769:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada60a16800 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571832.929772:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16800/0xef30c340bfb3d58d lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 6 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929778:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16800/0xef30c340bfb3d58d lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 6 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.929787:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16800/0xef30c340bfb3d58d lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 6 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930002:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571832.930008:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16800/0xef30c340bfb3d58d lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 6 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930017:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16800/0xef30c340bfb3d58d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 6 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930034:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16200/0xef30c340bfb3d594 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930041:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16200/0xef30c340bfb3d594 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930114:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16200/0xef30c340bfb3d594 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930120:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16200/0xef30c340bfb3d594 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930127:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16200/0xef30c340bfb3d594 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930132:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada60a16200 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571832.930136:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16200/0xef30c340bfb3d594 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 5 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930142:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16200/0xef30c340bfb3d594 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 5 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930154:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16200/0xef30c340bfb3d594 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 5 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930160:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571832.930163:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16200/0xef30c340bfb3d594 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 5 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930170:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16200/0xef30c340bfb3d594 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 5 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930185:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0c00/0xef30c340bfb3d59b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930191:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0c00/0xef30c340bfb3d59b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930197:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0c00/0xef30c340bfb3d59b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930203:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0c00/0xef30c340bfb3d59b lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930208:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0c00/0xef30c340bfb3d59b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930213:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada516e0c00 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571832.930216:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0c00/0xef30c340bfb3d59b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930222:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0c00/0xef30c340bfb3d59b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930232:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0c00/0xef30c340bfb3d59b lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930238:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571832.930241:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0c00/0xef30c340bfb3d59b lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930248:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0c00/0xef30c340bfb3d59b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 4 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930260:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1200/0xef30c340bfb3d5a2 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930266:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1200/0xef30c340bfb3d5a2 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930273:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1200/0xef30c340bfb3d5a2 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930279:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1200/0xef30c340bfb3d5a2 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930285:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1200/0xef30c340bfb3d5a2 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930290:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada516e1200 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571832.930293:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1200/0xef30c340bfb3d5a2 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 3 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930299:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1200/0xef30c340bfb3d5a2 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 3 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930308:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1200/0xef30c340bfb3d5a2 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 3 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930313:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571832.930316:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1200/0xef30c340bfb3d5a2 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930322:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1200/0xef30c340bfb3d5a2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930336:0:2385:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1400/0xef30c340bfb3d5a9 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930342:0:2385:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1400/0xef30c340bfb3d5a9 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930349:0:2385:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1400/0xef30c340bfb3d5a9 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930417:0:2385:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1400/0xef30c340bfb3d5a9 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930423:0:2385:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1400/0xef30c340bfb3d5a9 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930427:0:2385:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada516e1400 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571832.930431:0:2385:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1400/0xef30c340bfb3d5a9 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 2 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930436:0:2385:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1400/0xef30c340bfb3d5a9 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 2 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930446:0:2385:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1400/0xef30c340bfb3d5a9 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 2 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930451:0:2385:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571832.930454:0:2385:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1400/0xef30c340bfb3d5a9 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571832.930459:0:2385:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1400/0xef30c340bfb3d5a9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571832.930503:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a36a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613594752:192.168.202.113@tcp:63:ll_sa.0 00010000:00010000:2.0:1788571833.208240:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571834.236267:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788571835.267300:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571836.282996:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571837.303990:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788571837.816972:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a4bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449613594880:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571837.817068:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a49500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613595136:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571837.817096:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a4aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613595392:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571837.821917:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a49500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613595136:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571837.821938:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613595392:192.168.202.113@tcp:400:kworker.0 00010000:00010000:2.0:1788571838.352395:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788571838.447492:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613595776:192.168.202.113@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1788571838.447578:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613596032:192.168.202.113@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1788571838.447607:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1788571838.447614:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613595904:192.168.202.113@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1788571838.447640:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613596160:192.168.202.113@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1788571838.465656:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a48e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613596288:192.168.202.113@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1788571838.465675:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a49880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613596544:192.168.202.113@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1788571838.465690:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a49c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613596416:192.168.202.113@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1788571838.465699:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a49f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613596672:192.168.202.113@tcp:103:ldlm_bl.0 00000001:02000400:0.0:1788571842.251806:0:79924:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == replay-single test 34: abort recovery before client does replay (test mds_cleanup_orphans) ========================================================== 21:30:40 (1788571840) 00010000:00010000:0.0:1788571842.429614:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:0.0:1788571842.443747:0:79931:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571842.443777:0:79931:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571842.443839:0:79931:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616c00/0xef30c340bfb3d5b0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 79931 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571842.443850:0:79931:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616c00/0xef30c340bfb3d5b0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79931 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571842.443920:0:79931:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616c00/0xef30c340bfb3d5b0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79931 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571842.443951:0:79931:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f1880 pname:cluuid:pid:xid:nid:opc:job multiop:35eb2d60-2de0-4b00-9815-d5153edd0057:79931:1875449613596800:192.168.202.113@tcp:101:multiop.0 00000100:00100000:0.0:1788571842.444051:0:79931:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada51952c00 going to sleep for 11 seconds 00000100:00100000:0.0:1788571842.447551:0:79931:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 197568495646 gen 6 00000100:00100000:0.0:1788571842.447659:0:79931:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f1880 pname:cluuid:pid:xid:nid:opc:job multiop:35eb2d60-2de0-4b00-9815-d5153edd0057:79931:1875449613596800:192.168.202.113@tcp:101:multiop.0 00010000:00010000:0.0:1788571842.447682:0:79931:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616c00/0xef30c340bfb3d5b0 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7439 expref: -99 pid: 79931 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571842.447695:0:79931:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616c00/0xef30c340bfb3d5b0 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7439 expref: -99 pid: 79931 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571842.447706:0:79931:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616c00/0xef30c340bfb3d5b0 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7439 expref: -99 pid: 79931 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571842.447716:0:79931:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616c00/0xef30c340bfb3d5b0 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7439 expref: -99 pid: 79931 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571842.447727:0:79931:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616c00/0xef30c340bfb3d5b0 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7439 expref: -99 pid: 79931 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571842.447735:0:79931:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616c00/0xef30c340bfb3d5b0 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7439 expref: -99 pid: 79931 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571842.447742:0:79931:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616c00/0xef30c340bfb3d5b0 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7439 expref: -99 pid: 79931 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788571842.447754:0:79931:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada516f1880 x1875449613596800/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788571858 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788571842.447769:0:79931:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616c00/0xef30c340bfb3d5b0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7439 expref: -99 pid: 79931 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571842.447782:0:79931:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1788571842.447788:0:79931:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1788571842.447814:0:79931:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:0.0:1788571842.447820:0:79931:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d5b0 00000080:00010000:0.0:1788571842.447826:0:79931:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571842.447829:0:79931:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d5b0 from it ffffab0ec0d4bbf8 00010000:00010000:0.0:1788571842.447833:0:79931:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616c00/0xef30c340bfb3d5b0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7439 expref: -99 pid: 79931 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571842.447842:0:79931:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616c00/0xef30c340bfb3d5b0 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7439 expref: -99 pid: 79931 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571842.448824:0:79931:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:0.0:1788571842.448849:0:79931:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571842.448883:0:79931:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571842.448905:0:79931:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616c00/0xef30c340bfb3d5b0 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7439 expref: -99 pid: 79931 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571842.448918:0:79931:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616c00/0xef30c340bfb3d5b0 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7439 expref: -99 pid: 79931 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571842.448930:0:79931:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788571842.448936:0:79931:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571842.448939:0:79931:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d5b0 00000080:00010000:0.0:1788571842.448943:0:79931:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d5b0 from it ffffab0ec0d4bb38 00010000:00010000:0.0:1788571842.448946:0:79931:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616c00/0xef30c340bfb3d5b0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7439 expref: -99 pid: 79931 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571842.448954:0:79931:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616c00/0xef30c340bfb3d5b0 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7439 expref: -99 pid: 79931 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788571842.448963:0:79931:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:0.0:1788571842.448968:0:79931:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f34.replay-single, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), file ffff9ada504c5380, open_flags 8042, mode 81a4 opened 0 00000080:00200000:0.0:1788571842.448976:0:79931:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f34.replay-single, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000002:00010000:0.0:1788571842.449000:0:79931:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f34.replay-single,[0x200016b60:0xb:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0100103 00010000:00010000:0.0:1788571842.449012:0:79931:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616c00/0xef30c340bfb3d5b0 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b82f7439 expref: -99 pid: 79931 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571842.449058:0:79931:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616c00/0xef30c340bfb3d5b0 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f7439 expref: -99 pid: 79931 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571842.449066:0:79931:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1788571842.449069:0:79931:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616c00/0xef30c340bfb3d5b0 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f7439 expref: -99 pid: 79931 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571842.449251:0:79931:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616800/0xef30c340bfb3d5b7 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 79931 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571842.449263:0:79931:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616800/0xef30c340bfb3d5b7 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79931 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:0.0:1788571842.449275:0:79931:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:0.0:1788571842.449285:0:79931:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada516f1880 x1875449613597056/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:0.0:1788571842.449295:0:79931:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616800/0xef30c340bfb3d5b7 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79931 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571842.449318:0:79931:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f1880 pname:cluuid:pid:xid:nid:opc:job multiop:35eb2d60-2de0-4b00-9815-d5153edd0057:79931:1875449613597056:192.168.202.113@tcp:101:multiop.0 00000100:00100000:0.0:1788571842.449504:0:79931:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada51952c00 going to sleep for 11 seconds 00000100:00100000:0.0:1788571842.478453:0:79931:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f1880 pname:cluuid:pid:xid:nid:opc:job multiop:35eb2d60-2de0-4b00-9815-d5153edd0057:79931:1875449613597056:192.168.202.113@tcp:101:multiop.0 00010000:00010000:0.0:1788571842.478482:0:79931:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616800/0xef30c340bfb3d5b7 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7455 expref: -99 pid: 79931 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571842.478512:0:79931:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616800/0xef30c340bfb3d5b7 lrc: 4/0,1 mode: --/PR res: [0x200016b60:0xb:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7455 expref: -99 pid: 79931 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571842.478521:0:79931:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616800/0xef30c340bfb3d5b7 lrc: 4/0,1 mode: --/PR res: [0x200016b60:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7455 expref: -99 pid: 79931 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571842.478533:0:79931:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616800/0xef30c340bfb3d5b7 lrc: 4/0,1 mode: PR/PR res: [0x200016b60:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7455 expref: -99 pid: 79931 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571842.478544:0:79931:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616800/0xef30c340bfb3d5b7 lrc: 4/0,1 mode: PR/PR res: [0x200016b60:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7455 expref: -99 pid: 79931 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571842.478563:0:79931:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616800/0xef30c340bfb3d5b7 lrc: 5/1,1 mode: PR/PR res: [0x200016b60:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7455 expref: -99 pid: 79931 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571842.478570:0:79931:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616800/0xef30c340bfb3d5b7 lrc: 5/1,1 mode: PR/PR res: [0x200016b60:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7455 expref: -99 pid: 79931 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571842.478578:0:79931:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616800/0xef30c340bfb3d5b7 lrc: 4/1,0 mode: PR/PR res: [0x200016b60:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7455 expref: -99 pid: 79931 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788571842.478590:0:79931:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x37, status=0 req@ffff9ada516f1880 x1875449613597056/t197568495647(197568495647) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/664 e 0 to 0 dl 1788571858 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00000002:00100000:0.0:1788571842.478603:0:79931:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada516f1880 x1875449613597056/t197568495647(197568495647) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/664 e 0 to 0 dl 1788571858 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788571842.478618:0:79931:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616800/0xef30c340bfb3d5b7 lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7455 expref: -99 pid: 79931 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:0.0:1788571842.478632:0:79931:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616800/0xef30c340bfb3d5b7 lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7455 expref: -99 pid: 79931 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571842.478643:0:79931:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8590027616/11 (0 0) 00000080:00200000:0.0:1788571842.478650:0:79931:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1788571842.478662:0:79931:0:(namei.c:1628:ll_create_it()) VFS Op:name=f34.replay-single, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000080:00200000:0.0:1788571842.478692:0:79931:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200016b60:0xb:0x0](ffff9ada58217448) 00000080:00200000:0.0:1788571842.478761:0:79931:0:(file.c:6843:ll_layout_conf()) [0x200016b60:0xb:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:0.0:1788571842.478765:0:79931:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x200016b60:0xb:0x0](ffff9ada58217448) 00000080:00010000:0.0:1788571842.478769:0:79931:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200016b60:0xb:0x0](ffff9ada58217448) for lock 0xef30c340bfb3d5b7 00000080:00010000:0.0:1788571842.478778:0:79931:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200016b60:0xb:0x0](ffff9ada58217448) 00000080:00200008:0.0:1788571842.478792:0:79931:0:(file.c:1090:ll_file_open()) START file f34.replay-single:[0x200016b60:0xb:0x0](ffff9ada58217448), flags 100102 00000002:00100000:0.0:1788571842.478811:0:79931:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada516f1880 x1875449613597056/t197568495647(197568495647) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571858 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'multiop.0' uid:0 gid:0 projid:0 00000080:00010000:0.0:1788571842.478846:0:79931:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d5b7 from it ffff9ada51761600 00010000:00010000:0.0:1788571842.478850:0:79931:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616800/0xef30c340bfb3d5b7 lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f7455 expref: -99 pid: 79931 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571842.478876:0:79931:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616800/0xef30c340bfb3d5b7 lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f7455 expref: -99 pid: 79931 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571842.478909:0:79931:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200016b60:0xb:0x0](ffff9ada58217448), inode mode 81a4 mask 40 00000080:00200000:0.0:1788571842.492523:0:79937:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571842.492555:0:79937:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571842.492632:0:79937:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616c00/0xef30c340bfb3d5be lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 79937 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571842.492642:0:79937:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616c00/0xef30c340bfb3d5be lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79937 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571842.492652:0:79937:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616c00/0xef30c340bfb3d5be lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79937 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571842.492678:0:79937:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f2300 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:79937:1875449613597184:192.168.202.113@tcp:101:rm.0 00000100:00100000:0.0:1788571842.492774:0:79937:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada51952480 going to sleep for 11 seconds 00000080:00200000:2.0:1788571842.508121:0:79937:0:(file.c:1775:ll_merge_attr_nolock()) [0x200016b60:0xb:0x0] updating i_size 0 i_blocks 0 00000020:00010000:2.0:1788571842.508128:0:79937:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9ada51684250 (ffff9ada60b68f38/0) at cl_lock_release():245 00000020:00010000:2.0:1788571842.508132:0:79937:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9ada51684250 (ffff9ada60b68f38/0) at cl_lock_cancel():142 00000020:00010000:2.0:1788571842.508137:0:79937:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9ada51899f30 (ffff9ada60b687e8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1788571842.508141:0:79937:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9ada51684250 (ffff9ada60b68f38/0) at cl_lock_fini():72 00000020:00010000:2.0:1788571842.508144:0:79937:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9ada51899f30 (ffff9ada60b68f38/1) at cl_lock_fini():72 00000080:00200000:2.0:1788571842.508183:0:79937:0:(vvp_io.c:304:vvp_io_fini()) [0x200016b60:0xb:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1788571842.508239:0:79937:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571842.508263:0:79937:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571842.508279:0:79937:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616c00/0xef30c340bfb3d5be lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f746a expref: -99 pid: 79937 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571842.508291:0:79937:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616c00/0xef30c340bfb3d5be lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f746a expref: -99 pid: 79937 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571842.508299:0:79937:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571842.508306:0:79937:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571842.508309:0:79937:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d5be 00000080:00010000:2.0:1788571842.508315:0:79937:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d5be from it ffffab0ec146bc10 00010000:00010000:2.0:1788571842.508318:0:79937:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616c00/0xef30c340bfb3d5be lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f746a expref: -99 pid: 79937 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571842.508325:0:79937:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616c00/0xef30c340bfb3d5be lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f746a expref: -99 pid: 79937 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571842.508336:0:79937:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:2.0:1788571842.508345:0:79937:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f34.replay-single, flags=0 00000080:00200000:2.0:1788571842.508350:0:79937:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571842.508356:0:79937:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571842.508363:0:79937:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616c00/0xef30c340bfb3d5be lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f746a expref: -99 pid: 79937 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571842.508370:0:79937:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616c00/0xef30c340bfb3d5be lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f746a expref: -99 pid: 79937 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571842.508377:0:79937:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571842.508380:0:79937:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571842.508383:0:79937:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d5be 00000080:00010000:2.0:1788571842.508386:0:79937:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d5be from it ffffab0ec146bd68 00010000:00010000:2.0:1788571842.508389:0:79937:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616c00/0xef30c340bfb3d5be lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f746a expref: -99 pid: 79937 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571842.508395:0:79937:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616c00/0xef30c340bfb3d5be lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f746a expref: -99 pid: 79937 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1788571842.508401:0:79937:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:2.0:1788571842.508406:0:79937:0:(namei.c:2351:ll_unlink()) VFS Op:name=f34.replay-single, dir=[0x200000007:0x1:0x0](ffff9ada58125b88) 00010000:00010000:2.0:1788571842.508432:0:79937:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616c00/0xef30c340bfb3d5be lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b82f746a expref: -99 pid: 79937 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1788571842.508454:0:79937:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616800/0xef30c340bfb3d5b7 lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0xdb7edb52b82f7455 expref: -99 pid: 79931 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571842.508470:0:79937:0:(vvp_object.c:117:vvp_conf_set()) [0x200016b60:0xb:0x0]: losing layout lock 00010000:00010000:2.0:1788571842.508500:0:79937:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616800/0xef30c340bfb3d5b7 lrc: 1/0,0 mode: --/PR res: [0x200016b60:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0xdb7edb52b82f7455 expref: -99 pid: 79931 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571842.508509:0:79937:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616c00/0xef30c340bfb3d5be lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f746a expref: -99 pid: 79937 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1788571842.508514:0:79937:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:2.0:1788571842.508517:0:79937:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616800/0xef30c340bfb3d5b7 lrc: 0/0,0 mode: --/PR res: [0x200016b60:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0xdb7edb52b82f7455 expref: -99 pid: 79931 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571842.508533:0:79937:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616c00/0xef30c340bfb3d5be lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f746a expref: -99 pid: 79937 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:2.0:1788571842.508551:0:79937:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1788571842.508559:0:79937:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada60a36a00 x1875449613597568/t0(0) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788571842.508595:0:79937:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a36a00 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:79937:1875449613597568:192.168.202.113@tcp:36:rm.0 00000100:00100000:2.0:1788571842.508669:0:79937:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada429ad180 going to sleep for 11 seconds 00000100:00100000:0.0:1788571842.518163:0:79937:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a36a00 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:79937:1875449613597568:192.168.202.113@tcp:36:rm.0 00000100:00100000:2.0:1788571842.935237:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a34380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613597824:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571842.935323:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1788571842.935330:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a36d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449613597696:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571842.935355:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a34e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613597952:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571842.941265:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a34e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613597952:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571843.554653:0:79948:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 197568495648 gen 6 00000100:00100000:2.0:1788571843.554672:0:79948:0:(client.c:3175:ptlrpc_free_committed()) @@@ keeping (FL_REPLAY) req@ffff9ada516f1880 x1875449613597056/t197568495647(197568495647) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571858 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00000100:00100000:2.0:1788571843.554697:0:79948:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4ad80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:79948:1875449613598080:192.168.202.113@tcp:101:lfs.0 00010000:00010000:2.0:1788571843.554720:0:79948:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51762000/0xef30c340bfb3d5cc lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7494 expref: -99 pid: 79948 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571843.554733:0:79948:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51762000/0xef30c340bfb3d5cc lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7494 expref: -99 pid: 79948 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571843.554746:0:79948:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51762000/0xef30c340bfb3d5cc lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7494 expref: -99 pid: 79948 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571843.554757:0:79948:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51762000/0xef30c340bfb3d5cc lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7494 expref: -99 pid: 79948 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571843.554771:0:79948:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51762000/0xef30c340bfb3d5cc lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7494 expref: -99 pid: 79948 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571843.554779:0:79948:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51762000/0xef30c340bfb3d5cc lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7494 expref: -99 pid: 79948 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571843.554787:0:79948:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51762000/0xef30c340bfb3d5cc lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7494 expref: -99 pid: 79948 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1788571843.554809:0:79948:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada60a4ad80 x1875449613598080/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788571859 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1788571843.554824:0:79948:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51762000/0xef30c340bfb3d5cc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7494 expref: -99 pid: 79948 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571843.554838:0:79948:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:2.0:1788571843.554845:0:79948:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:2.0:1788571843.554920:0:79948:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:2.0:1788571843.554927:0:79948:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d5cc 00000080:00010000:2.0:1788571843.554932:0:79948:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571843.554935:0:79948:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d5cc from it ffffab0ec0dd3cb8 00010000:00010000:2.0:1788571843.554938:0:79948:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51762000/0xef30c340bfb3d5cc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7494 expref: -99 pid: 79948 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571843.554946:0:79948:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51762000/0xef30c340bfb3d5cc lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7494 expref: -99 pid: 79948 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000100:00100000:0.0:1788571843.571501:0:79948:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4bb80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:79948:1875449613598208:192.168.202.113@tcp:41:lfs.0 00800000:00000004:0.0:1788571843.571604:0:79948:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: update ffff9ada4481d198 cache blocks 317372/353529 objects 1023691/1024000 00000080:00200080:0.0:1788571843.571747:0:79948:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02b79a30 00800000:00000080:0.0:1788571843.571766:0:79948:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00800000:00000004:0.0:1788571843.571773:0:79948:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: age 3740, max_age 3760 00000100:00100000:0.0:1788571843.571821:0:79948:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0e00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:79948:1875449613598336:192.168.202.113@tcp:41:lfs.0 00000100:00100000:0.0:1788571843.571936:0:79948:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada51952b40 going to sleep for 11 seconds 00000100:00100000:0.0:1788571843.574595:0:79948:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0e00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:79948:1875449613598336:192.168.202.113@tcp:41:lfs.0 00800000:00000004:0.0:1788571843.574631:0:79948:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: update ffff9ada44819e18 cache blocks 317416/353529 objects 1023739/1024000 00000080:00200000:2.0:1788571843.625481:0:79955:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571843.625514:0:79955:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571843.625598:0:79955:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51762000/0xef30c340bfb3d5cc lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7494 expref: -99 pid: 79948 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1788571843.625615:0:79955:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51762000/0xef30c340bfb3d5cc lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7494 expref: -99 pid: 79948 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1788571843.625624:0:79955:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571843.625633:0:79955:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571843.625637:0:79955:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d5cc 00000080:00010000:2.0:1788571843.625645:0:79955:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d5cc from it ffffab0ec16d3bd0 00010000:00010000:2.0:1788571843.625650:0:79955:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51762000/0xef30c340bfb3d5cc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7494 expref: -99 pid: 79948 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1788571843.625659:0:79955:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51762000/0xef30c340bfb3d5cc lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7494 expref: -99 pid: 79948 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:2.0:1788571843.625674:0:79955:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:2.0:1788571843.625688:0:79955:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=fsa-oleg213-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), flags=1536 00000080:00200000:2.0:1788571843.625692:0:79955:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571843.625699:0:79955:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571843.625706:0:79955:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51762000/0xef30c340bfb3d5cc lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7494 expref: -99 pid: 79948 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:2.0:1788571843.625715:0:79955:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51762000/0xef30c340bfb3d5cc lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7494 expref: -99 pid: 79948 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:2.0:1788571843.625721:0:79955:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571843.625726:0:79955:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571843.625728:0:79955:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d5cc 00000080:00010000:2.0:1788571843.625732:0:79955:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d5cc from it ffffab0ec16d3cd0 00010000:00010000:2.0:1788571843.625735:0:79955:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51762000/0xef30c340bfb3d5cc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7494 expref: -99 pid: 79948 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:2.0:1788571843.625743:0:79955:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51762000/0xef30c340bfb3d5cc lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7494 expref: -99 pid: 79948 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:2.0:1788571843.625750:0:79955:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:2.0:1788571843.625756:0:79955:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571843.625763:0:79955:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571843.625769:0:79955:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51762000/0xef30c340bfb3d5cc lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7494 expref: -99 pid: 79948 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:2.0:1788571843.625777:0:79955:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51762000/0xef30c340bfb3d5cc lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7494 expref: -99 pid: 79948 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:2.0:1788571843.625784:0:79955:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571843.625788:0:79955:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571843.625790:0:79955:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d5cc 00000080:00010000:2.0:1788571843.625793:0:79955:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d5cc from it ffffab0ec16d3db0 00010000:00010000:2.0:1788571843.625796:0:79955:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51762000/0xef30c340bfb3d5cc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7494 expref: -99 pid: 79948 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:2.0:1788571843.625804:0:79955:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51762000/0xef30c340bfb3d5cc lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7494 expref: -99 pid: 79948 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000080:00200000:2.0:1788571843.625812:0:79955:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:2.0:1788571843.625817:0:79955:0:(namei.c:2062:ll_create_nd()) VFS Op:name=fsa-oleg213-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), flags=33188, excl=1 00000080:00200000:2.0:1788571843.625821:0:79955:0:(namei.c:2017:ll_mknod()) VFS Op:name=fsa-oleg213-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9ada58125b88) mode 100644 dev 0 00010000:00010000:2.0:1788571843.625847:0:79955:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51762000/0xef30c340bfb3d5cc lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b82f7494 expref: -99 pid: 79948 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:2.0:1788571843.626099:0:79955:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51762000/0xef30c340bfb3d5cc lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f7494 expref: -99 pid: 79948 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:2.0:1788571843.626106:0:79955:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1788571843.626108:0:79955:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51762000/0xef30c340bfb3d5cc lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f7494 expref: -99 pid: 79948 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000020:00100000:2.0:1788571843.626137:0:79955:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1788571843.626222:0:79955:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada60a36a00 x1875449613598848/t0(0) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788571843.626249:0:79955:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a36a00 pname:cluuid:pid:xid:nid:opc:job mcreate:35eb2d60-2de0-4b00-9815-d5153edd0057:79955:1875449613598848:192.168.202.113@tcp:36:mcreate.0 00000100:00100000:2.0:1788571843.626420:0:79955:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada429ade40 going to sleep for 11 seconds 00000100:00100000:2.0:1788571843.635488:0:79955:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a36a00 pname:cluuid:pid:xid:nid:opc:job mcreate:35eb2d60-2de0-4b00-9815-d5153edd0057:79955:1875449613598848:192.168.202.113@tcp:36:mcreate.0 00000080:00200000:2.0:1788571843.635548:0:79955:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200016b60:0xc:0x0](ffff9ada58191988) 00000080:00200000:2.0:1788571843.635588:0:79955:0:(namei.c:2070:ll_create_nd()) VFS Op:name=fsa-oleg213-client.virtnet, unhashed 1 00000100:00100000:2.0:1788571843.672015:0:79958:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a48000 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:79958:1875449613598976:192.168.202.113@tcp:101:rm.0 00010000:00010000:2.0:1788571843.672044:0:79958:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5d3 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f74b0 expref: -99 pid: 79958 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571843.672056:0:79958:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5d3 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f74b0 expref: -99 pid: 79958 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571843.672066:0:79958:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5d3 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f74b0 expref: -99 pid: 79958 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571843.672076:0:79958:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5d3 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f74b0 expref: -99 pid: 79958 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571843.672090:0:79958:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5d3 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f74b0 expref: -99 pid: 79958 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571843.672097:0:79958:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5d3 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f74b0 expref: -99 pid: 79958 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571843.672105:0:79958:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5d3 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f74b0 expref: -99 pid: 79958 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1788571843.672116:0:79958:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada60a48000 x1875449613598976/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788571859 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1788571843.672129:0:79958:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5d3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f74b0 expref: -99 pid: 79958 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571843.672141:0:79958:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:2.0:1788571843.672159:0:79958:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:2.0:1788571843.672184:0:79958:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:2.0:1788571843.672190:0:79958:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d5d3 00000080:00010000:2.0:1788571843.672196:0:79958:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571843.672200:0:79958:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d5d3 from it ffffab0ec16d3b58 00010000:00010000:2.0:1788571843.672203:0:79958:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5d3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f74b0 expref: -99 pid: 79958 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571843.672213:0:79958:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5d3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f74b0 expref: -99 pid: 79958 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571843.672237:0:79958:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:2.0:1788571843.672256:0:79958:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=fsa-oleg213-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), flags=8 00000080:00200000:2.0:1788571843.672261:0:79958:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=fsa-oleg213-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=getattr 00000002:00010000:2.0:1788571843.672283:0:79958:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: fsa-oleg213-client.virtnet,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 01 00010000:00010000:2.0:1788571843.672323:0:79958:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d5da lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 79958 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571843.672332:0:79958:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d5da lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 79958 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571843.672341:0:79958:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d5da lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 79958 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571843.672682:0:79958:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a34e00 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:79958:1875449613599104:192.168.202.113@tcp:101:rm.0 00000100:00100000:2.0:1788571843.672768:0:79958:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada429ade40 going to sleep for 11 seconds 00000100:00100000:2.0:1788571843.679171:0:79958:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a34e00 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:79958:1875449613599104:192.168.202.113@tcp:101:rm.0 00010000:00010000:2.0:1788571843.679200:0:79958:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d5da lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f74cc expref: -99 pid: 79958 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571843.679221:0:79958:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d5da lrc: 4/1,0 mode: --/PR res: [0x200016b60:0xc:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f74cc expref: -99 pid: 79958 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571843.679230:0:79958:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d5da lrc: 4/1,0 mode: --/PR res: [0x200016b60:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f74cc expref: -99 pid: 79958 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571843.679241:0:79958:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d5da lrc: 4/1,0 mode: PR/PR res: [0x200016b60:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f74cc expref: -99 pid: 79958 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571843.679253:0:79958:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d5da lrc: 4/1,0 mode: PR/PR res: [0x200016b60:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f74cc expref: -99 pid: 79958 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571843.679264:0:79958:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d5da lrc: 5/2,0 mode: PR/PR res: [0x200016b60:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f74cc expref: -99 pid: 79958 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571843.679276:0:79958:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d5da lrc: 5/2,0 mode: PR/PR res: [0x200016b60:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f74cc expref: -99 pid: 79958 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571843.679282:0:79958:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d5da lrc: 4/1,0 mode: PR/PR res: [0x200016b60:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f74cc expref: -99 pid: 79958 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1788571843.679292:0:79958:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada60a34e00 x1875449613599104/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 600/584 e 0 to 0 dl 1788571859 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1788571843.679306:0:79958:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d5da lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f74cc expref: -99 pid: 79958 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571843.679316:0:79958:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8590027616/12 (0 0) 00000080:00200000:2.0:1788571843.679321:0:79958:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1788571843.679341:0:79958:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200016b60:0xc:0x0], updating i_size 0 00000080:00200000:2.0:1788571843.679343:0:79958:0:(namei.c:147:ll_iget()) got inode: [0x200016b60:0xc:0x0](ffff9ada58191988): rc = 0 00000080:00200000:2.0:1788571843.679374:0:79958:0:(file.c:6843:ll_layout_conf()) [0x200016b60:0xc:0x0]: layout version change: 4294967294 -> 4294967295 00000080:00010000:2.0:1788571843.679381:0:79958:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200016b60:0xc:0x0](ffff9ada58191988) for lock 0xef30c340bfb3d5da 00000080:00010000:2.0:1788571843.679391:0:79958:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200016b60:0xc:0x0](ffff9ada58191988) 00000080:00010000:2.0:1788571843.679394:0:79958:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d5da from it ffffab0ec16d3b20 00010000:00010000:2.0:1788571843.679397:0:79958:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d5da lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f74cc expref: -99 pid: 79958 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571843.679406:0:79958:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d5da lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f74cc expref: -99 pid: 79958 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:2.0:1788571843.679444:0:79958:0:(file.c:6296:ll_getattr_dentry()) START file fsa-oleg213-client.virtnet:[0x200016b60:0xc:0x0](ffff9ada58191988), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1788571843.679451:0:79958:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200016b60:0xc:0x0](ffff9ada58191988),name=fsa-oleg213-client.virtnet 00000002:00010000:2.0:1788571843.679470:0:79958:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200016b60:0xc:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571843.679478:0:79958:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d5da lrc: 2/1,0 mode: PR/PR res: [0x200016b60:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f74cc expref: -99 pid: 79958 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571843.679487:0:79958:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d5da lrc: 2/1,0 mode: PR/PR res: [0x200016b60:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f74cc expref: -99 pid: 79958 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571843.679494:0:79958:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571843.679498:0:79958:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200016b60:0xc:0x0](ffff9ada58191988) 00000080:00010000:2.0:1788571843.679501:0:79958:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200016b60:0xc:0x0](ffff9ada58191988) for lock 0xef30c340bfb3d5da 00000080:00010000:2.0:1788571843.679504:0:79958:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d5da from it ffffab0ec16d3cb8 00010000:00010000:2.0:1788571843.679507:0:79958:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d5da lrc: 3/1,0 mode: PR/PR res: [0x200016b60:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f74cc expref: -99 pid: 79958 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571843.679513:0:79958:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d5da lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f74cc expref: -99 pid: 79958 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571843.679576:0:79958:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571843.679584:0:79958:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571843.679590:0:79958:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5d3 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f74b0 expref: -99 pid: 79958 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571843.681239:0:79958:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5d3 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f74b0 expref: -99 pid: 79958 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571843.681258:0:79958:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571843.681266:0:79958:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571843.681269:0:79958:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d5d3 00000080:00010000:2.0:1788571843.681276:0:79958:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d5d3 from it ffffab0ec16d3c10 00010000:00010000:2.0:1788571843.681279:0:79958:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5d3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f74b0 expref: -99 pid: 79958 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571843.681288:0:79958:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5d3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f74b0 expref: -99 pid: 79958 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571843.681304:0:79958:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:2.0:1788571843.681318:0:79958:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=fsa-oleg213-client.virtnet, flags=0 00000080:00200000:2.0:1788571843.681323:0:79958:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571843.681339:0:79958:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571843.681361:0:79958:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5d3 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f74b0 expref: -99 pid: 79958 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571843.681368:0:79958:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5d3 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f74b0 expref: -99 pid: 79958 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571843.681374:0:79958:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571843.681377:0:79958:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571843.681380:0:79958:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d5d3 00000080:00010000:2.0:1788571843.681382:0:79958:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d5d3 from it ffffab0ec16d3d68 00010000:00010000:2.0:1788571843.681385:0:79958:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5d3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f74b0 expref: -99 pid: 79958 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571843.681391:0:79958:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5d3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f74b0 expref: -99 pid: 79958 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1788571843.681403:0:79958:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:2.0:1788571843.681408:0:79958:0:(namei.c:2351:ll_unlink()) VFS Op:name=fsa-oleg213-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9ada58125b88) 00010000:00010000:2.0:1788571843.681432:0:79958:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5d3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b82f74b0 expref: -99 pid: 79958 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1788571843.681456:0:79958:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d5da lrc: 2/0,0 mode: PR/PR res: [0x200016b60:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0xdb7edb52b82f74cc expref: -99 pid: 79958 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571843.681465:0:79958:0:(vvp_object.c:117:vvp_conf_set()) [0x200016b60:0xc:0x0]: losing layout lock 00010000:00010000:2.0:1788571843.681500:0:79958:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d5da lrc: 1/0,0 mode: --/PR res: [0x200016b60:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0xdb7edb52b82f74cc expref: -99 pid: 79958 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571843.681508:0:79958:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5d3 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f74b0 expref: -99 pid: 79958 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1788571843.681526:0:79958:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:2.0:1788571843.681529:0:79958:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d5da lrc: 0/0,0 mode: --/PR res: [0x200016b60:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0xdb7edb52b82f74cc expref: -99 pid: 79958 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571843.681544:0:79958:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5d3 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f74b0 expref: -99 pid: 79958 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:2.0:1788571843.681563:0:79958:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1788571843.681573:0:79958:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada60a34e00 x1875449613599360/t0(0) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 512/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788571843.682789:0:79958:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a34e00 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:79958:1875449613599360:192.168.202.113@tcp:36:rm.0 00000100:00100000:2.0:1788571843.683009:0:79958:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada429ade40 going to sleep for 11 seconds 00000100:00100000:2.0:1788571843.698943:0:79958:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a34e00 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:79958:1875449613599360:192.168.202.113@tcp:36:rm.0 00000080:00200000:2.0:1788571843.699011:0:79958:0:(vvp_io.c:1826:vvp_io_init()) [0x200016b60:0xc:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:2.0:1788571843.699021:0:79958:0:(vvp_io.c:304:vvp_io_fini()) [0x200016b60:0xc:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1788571843.699029:0:79958:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200016b60:0xc:0x0](ffff9ada58191988) 00010000:00010000:0.0:1788571845.495401:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788571848.061056:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613599616:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571848.064768:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a35c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613600000:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571848.064789:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a34a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613599872:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571848.064801:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a35880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613599744:192.168.202.113@tcp:400:kworker.0 00010000:00010000:0.0:1788571848.567303:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788571851.639716:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788571853.181442:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449613600768:192.168.202.113@tcp:38:ptlrpcd_00_01.0 00000100:00080000:0.0:1788571853.192179:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788571853.192195:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-19) 00000100:00100000:0.0:1788571853.192201:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449613600768:192.168.202.113@tcp:38:ptlrpcd_00_01.0 00010000:00010000:0.0:1788571854.711174:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788571858.296541:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788571858.296552:0:58077:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788571858.296557:0:58077:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788571858.296569:0:58077:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 3770 00000100:00080000:0.0:1788571858.296583:0:58077:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 5s 00000100:00080000:0.0:1788571858.296592:0:58077:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788571858.296605:0:58077:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788571858.296627:0:58077:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada516f0700 x1875449613601024/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788571858.296702:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613601152:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571858.296740:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613601408:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571858.297047:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449613601024:192.168.202.113@tcp:38:kworker.0 00000100:00080000:2.0:1788571858.304522:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788571858.304530:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-19) 00000100:00100000:2.0:1788571858.304534:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449613601024:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788571858.305091:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613601152:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571858.305108:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613601408:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571863.417609:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449613601536:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571863.417622:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449613601664:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788571863.417731:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613601920:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571863.417767:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1788571863.417776:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613601792:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571863.417802:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613602048:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571863.435664:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613601792:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571863.435687:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613602048:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571863.435700:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613601920:192.168.202.113@tcp:400:kworker.0 00000100:00080000:2.0:1788571863.444456:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788571863.444488:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-19) 00000100:00100000:2.0:1788571863.444494:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449613601664:192.168.202.113@tcp:38:kworker.0 00010000:00010000:0.0:1788571867.000372:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788571868.536566:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a4bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449613602304:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788571868.536626:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a48000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449613602176:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571868.536676:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a4b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613602560:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571868.536712:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1788571868.536720:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a4ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613602432:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571868.536744:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a4a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613602688:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571868.540390:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613602432:192.168.202.113@tcp:400:kworker.0 00000100:00080000:0.0:1788571868.544663:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788571868.544677:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-19) 00000100:00100000:0.0:1788571868.544682:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449613602304:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788571868.547533:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613602560:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571868.547585:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613602688:192.168.202.113@tcp:400:kworker.0 00000100:00000400:0.0:1788571869.559176:0:2384:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788571853/real 1788571853] req@ffff9ada603f7100 x1875449613600128/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 1 dl 1788571869 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788571869.559225:0:2384:0:(client.c:2548:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9ada603f7100 x1875449613600128/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 1 dl 1788571869 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788571869.559237:0:2384:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from FULL to DISCONN 00000100:02020000:0.0:1788571869.559242:0:2384:0:(import.c:186:ptlrpc_set_import_discon()) MGC192.168.202.113@tcp: Connection to MGS (at 192.168.202.113@tcp) was lost; in progress operations using this service will fail 00000100:00080000:0.0:1788571869.559245:0:2384:0:(import.c:192:ptlrpc_set_import_discon()) import MGS@192.168.202.113@tcp for MGC192.168.202.113@tcp not replayable, auto-deactivating 00000100:00080000:0.0:1788571869.559248:0:2384:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import MGS INVALID 00000100:00100000:0.0:1788571869.559252:0:2384:0:(client.c:3591:ptlrpc_abort_inflight()) @@@ inflight req@ffff9ada603f7100 x1875449613600128/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 1 dl 1788571869 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788571869.559262:0:2384:0:(client.c:3591:ptlrpc_abort_inflight()) @@@ inflight req@ffff9ada516f0e00 x1875449613600896/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788571874 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788571869.559285:0:2384:0:(client.c:3591:ptlrpc_abort_inflight()) @@@ inflight req@ffff9ada603f7b80 x1875449613601536/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788571879 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788571869.559294:0:2384:0:(client.c:3591:ptlrpc_abort_inflight()) @@@ inflight req@ffff9ada60a48000 x1875449613602176/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788571884 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:0.0:1788571869.559315:0:2384:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808001 00000100:00080000:0.0:1788571869.559318:0:2384:0:(pinger.c:193:ptlrpc_pinger_ir_down()) IR down 10000000:01000000:0.0:1788571869.559320:0:2384:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808002 00000100:00080000:0.0:1788571869.559323:0:2384:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00100000:0.0:1788571869.559373:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449613601536:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571869.559383:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449613600896:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571869.559388:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449613600128:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571869.560440:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a48000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449613602176:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571869.561051:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a49180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449613602816:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788571869.561144:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a49f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449613602944:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788571869.565742:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a49c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613603072:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571869.565768:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a48e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613603328:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571869.565781:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a49880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613603200:192.168.202.113@tcp:400:kworker.0 00000100:00080000:0.0:1788571869.565981:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788571869.565989:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-19) 00000100:00100000:0.0:1788571869.565993:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a49f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449613602944:192.168.202.113@tcp:38:kworker.0 00010000:00010000:0.0:1788571870.071533:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571872.631110:0:79358:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d5e1 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571872.631131:0:79358:0:(ldlm_request.c:1275:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d5e1 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571873.143176:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788571874.679382:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449613603584:192.168.202.113@tcp:38:kworker.0 00010000:00010000:2.0:1788571874.687110:0:15205:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 0 locks of 0 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1788571874.687140:0:15205:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 0 locks of 0 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1788571874.687145:0:15205:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 0 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1788571874.687150:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571874.687152:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571874.687154:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788571874.696014:0:2378:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.113@tcp (source: 12345-192.168.202.113@tcp) 00000100:00100000:2.0:1788571874.696126:0:15197:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875449638547840 00000100:00100000:2.0:1788571874.696135:0:15197:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875449638547840 00000100:00100000:2.0:1788571874.696189:0:15197:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.113@tcp, seq: 235 00000100:00100000:2.0:1788571874.697844:0:15197:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9ada60a34a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_001:LOV_OSC_UUID+4:12091:x1875449638547840:12345-192.168.202.113@tcp:104: 00010000:00010000:2.0:1788571874.697883:0:15197:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 000000009dd3094c/0xef30c340bfb3d5c5 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x607:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000010000 nid: local remote: 0xdb7edb52b82f7471 expref: -99 pid: 79937 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571874.697978:0:15197:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1788571874.697987:0:15197:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9ada60a34a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_001:LOV_OSC_UUID+4:12091:x1875449638547840:12345-192.168.202.113@tcp:104: Request processed in 1788us (1982us total) trans 0 rc 0/0 00000100:00100000:2.0:1788571874.697996:0:15197:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.113@tcp, seq: 235 00010000:00010000:2.0:1788571874.698147:0:15205:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1788571874.698154:0:15205:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 000000009dd3094c/0xef30c340bfb3d5c5 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x607:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000010000 nid: local remote: 0xdb7edb52b82f7471 expref: -99 pid: 79937 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571874.698162:0:15205:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada53016c00 already unused, calling callback (ffffffffc102e300) 00010000:00010000:2.0:1788571874.698168:0:15205:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 000000009dd3094c/0xef30c340bfb3d5c5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x607:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800428400010000 nid: local remote: 0xdb7edb52b82f7471 expref: -99 pid: 79937 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571874.698187:0:15205:0:(vvp_io.c:1826:vvp_io_init()) [0x200016b60:0xb:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:2.0:1788571874.698201:0:15205:0:(vvp_io.c:304:vvp_io_fini()) [0x200016b60:0xb:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:2.0:1788571874.698240:0:15205:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 000000009dd3094c/0xef30c340bfb3d5c5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x607:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400010020 nid: local remote: 0xdb7edb52b82f7471 expref: -99 pid: 79937 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571874.698247:0:15205:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1788571874.698268:0:15205:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 000000009dd3094c/0xef30c340bfb3d5c5 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x607:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400010020 nid: local remote: 0xdb7edb52b82f7471 expref: -99 pid: 79937 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571874.698277:0:15205:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 000000009dd3094c/0xef30c340bfb3d5c5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x607:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400010020 nid: local remote: 0xdb7edb52b82f7471 expref: -99 pid: 79937 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571874.698298:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571874.698301:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571874.698303:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788571874.698316:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613604096:192.168.202.113@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788571874.699768:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788571874.707935:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a35c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613604096:192.168.202.113@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1788571874.722046:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449613603584:192.168.202.113@tcp:38:kworker.0 00000100:00080000:0.0:1788571874.725989:0:80286:0:(import.c:1642:ptlrpc_invalidate_import_thread()) thread invalidate import lustre-MDT0000-mdc-ffff9ada58b0a000 to lustre-MDT0000_UUID@192.168.202.113@tcp 00000100:00080000:0.0:1788571874.725997:0:80286:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import lustre-MDT0000_UUID INVALID 00000100:00100000:0.0:1788571874.726007:0:80286:0:(client.c:3603:ptlrpc_abort_inflight()) @@@ aborting waiting req req@ffff9ada60a35880 x1875449613603456/t0(0) o63->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:24/4 lens 680/4320 e 0 to 0 dl 0 ref 1 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'ll_sa.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788571874.726033:0:80286:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 197568495648 gen 7 00000100:00100000:0.0:1788571874.726039:0:80286:0:(client.c:3164:ptlrpc_free_committed()) @@@ free request with old gen req@ffff9ada60a36a00 x1875449613598848/t197568495649(197568495649) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788571859 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788571874.726063:0:80286:0:(client.c:3164:ptlrpc_free_committed()) @@@ free request with old gen req@ffff9ada60a34e00 x1875449613599360/t197568495650(197568495650) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 512/456 e 0 to 0 dl 1788571859 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788571874.726077:0:80286:0:(client.c:3214:ptlrpc_free_committed()) @@@ free stale open request req@ffff9ada516f1880 x1875449613597056/t197568495647(197568495647) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571858 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00000100:00100000:0.0:1788571874.726099:0:80286:0:(import.c:323:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 00010000:00010000:2.0:1788571874.726189:0:2386:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d5e1 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x0 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571874.726201:0:2386:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d5e1 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571874.726208:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d5e1 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571874.726215:0:2386:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d5e1 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571874.726223:0:2386:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d5e1 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571874.726229:0:2386:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada516e0000 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571874.726233:0:2386:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d5e1 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 2 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571874.726240:0:2386:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d5e1 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 2 type: IBT flags: 0x206400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571874.726247:0:2386:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d5e1 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 2 type: IBT flags: 0x20e400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571874.726254:0:2386:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571874.726261:0:2386:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d5e1 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571874.726268:0:2386:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d5e1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x3/0x3 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 79358 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571874.726305:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a35880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:0:1875449613603456:192.168.202.113@tcp:63:ll_sa.0 00000100:00080000:0.0:1788571874.736173:0:80286:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from EVICTED to RECOVER 00000100:00080000:0.0:1788571874.736187:0:80286:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from RECOVER to FULL 00000100:02000000:0.0:1788571874.738155:0:80286:0:(import.c:1787:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff9ada58b0a000: Connection restored to 192.168.202.113@tcp (at 192.168.202.113@tcp) 00010000:00010000:2.0:1788571876.215476:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788571877.242569:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571879.287109:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788571879.811475:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613604480:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571879.811511:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613604608:192.168.202.113@tcp:400:kworker.0 00010000:00080000:0.0:1788571879.814349:0:2384:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff9ada603f6680 x1875449613604224/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788571895 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788571879.814377:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613604224:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571879.814396:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613604352:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571879.863005:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788571869/real 1788571869] req@ffff9ada60a49180 x1875449613602816/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788571879 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788571879.863052:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60a49180 x1875449613602816/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788571879 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788571879.863078:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788571879.863110:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788571879.863115:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a49180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449613602816:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788571879.863494:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449613604736:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788571879.877707:0:2382:0:(import.c:1176:ptlrpc_connect_interpret()) MGC192.168.202.113@tcp: connect to target with instance 0 10000000:01000000:0.0:1788571879.877731:0:2382:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808005 00000100:02000400:0.0:1788571879.877742:0:2382:0:(import.c:1328:ptlrpc_connect_interpret()) Evicted from MGS (at 192.168.202.113@tcp) after server handle changed from 0xdb7edb52b82f6eba to 0xdb7edb52b82f7892 00000100:00080000:0.0:1788571879.877748:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to EVICTED 00000100:00080000:0.0:1788571879.877755:0:2382:0:(import.c:1718:ptlrpc_import_recovery_state_machine()) evicted from MGS@192.168.202.113@tcp; invalidating 00000100:00080000:0.0:1788571879.885250:0:80311:0:(import.c:1642:ptlrpc_invalidate_import_thread()) thread invalidate import MGC192.168.202.113@tcp to MGS@192.168.202.113@tcp 00000100:00100000:0.0:1788571879.885258:0:80311:0:(import.c:323:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 10000000:01000000:0.0:1788571879.885263:0:80311:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808003 00010000:00010000:0.0:1788571879.885292:0:80311:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.202.113@tcp lock: ffff9ada5163ee00/0xef30c340bfb3d2c3 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x110e400000000 nid: local remote: 0xdb7edb52b82f6f8c expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1788571879.885306:0:80311:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.202.113@tcp lock: ffff9ada5163ee00/0xef30c340bfb3d2c3 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x110f400000000 nid: local remote: 0xdb7edb52b82f6f8c expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1788571879.885313:0:80311:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:0.0:1788571879.885332:0:80311:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1788571879.885344:0:80311:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.202.113@tcp lock: ffff9ada5163ee00/0xef30c340bfb3d2c3 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x590f400000000 nid: local remote: 0xdb7edb52b82f6f8c expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571879.885366:0:80311:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.202.113@tcp lock: ffff9ada5163fa00/0xef30c340bfb3d2d1 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0xdb7edb52b82f6f9a expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1788571879.885373:0:80311:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.202.113@tcp lock: ffff9ada5163fa00/0xef30c340bfb3d2d1 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0xdb7edb52b82f6f9a expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1788571879.885377:0:80311:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:0.0:1788571879.885381:0:80311:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1788571879.885384:0:80311:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.202.113@tcp lock: ffff9ada5163fa00/0xef30c340bfb3d2d1 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0xdb7edb52b82f6f9a expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571879.885401:0:80311:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.202.113@tcp lock: ffff9ada5163ea00/0xef30c340bfb3d2bc lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0xdb7edb52b82f6f85 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1788571879.885407:0:80311:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.202.113@tcp lock: ffff9ada5163ea00/0xef30c340bfb3d2bc lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0xdb7edb52b82f6f85 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1788571879.885412:0:80311:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:0.0:1788571879.885416:0:80311:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1788571879.885419:0:80311:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.202.113@tcp lock: ffff9ada5163ea00/0xef30c340bfb3d2bc lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0xdb7edb52b82f6f85 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571879.885433:0:80311:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.202.113@tcp lock: ffff9ada5163f400/0xef30c340bfb3d2ca lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0xdb7edb52b82f6f93 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1788571879.885440:0:80311:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.202.113@tcp lock: ffff9ada5163f400/0xef30c340bfb3d2ca lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0xdb7edb52b82f6f93 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1788571879.885444:0:80311:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:0.0:1788571879.885447:0:80311:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1788571879.885449:0:80311:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.202.113@tcp lock: ffff9ada5163f400/0xef30c340bfb3d2ca lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0xdb7edb52b82f6f93 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1788571879.885463:0:80311:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from EVICTED to RECOVER 00000100:00080000:0.0:1788571879.885468:0:80311:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from RECOVER to FULL 10000000:01000000:0.0:1788571879.885471:0:80311:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808004 00000100:00080000:0.0:1788571879.885474:0:80311:0:(pinger.c:186:ptlrpc_pinger_ir_up()) IR up 00000100:02000000:0.0:1788571879.885479:0:80311:0:(import.c:1787:ptlrpc_import_recovery_state_machine()) MGC192.168.202.113@tcp: Connection restored to 192.168.202.113@tcp (at 192.168.202.113@tcp) 00010000:00010000:0.0:1788571880.312742:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788571881.085502:0:80337:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a49180 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:80337:1875449613604864:192.168.202.113@tcp:101:lfs.0 00010000:00010000:0.0:1788571881.085541:0:80337:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5e8 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f78a0 expref: -99 pid: 80337 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571881.085556:0:80337:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5e8 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f78a0 expref: -99 pid: 80337 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571881.085570:0:80337:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5e8 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f78a0 expref: -99 pid: 80337 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571881.085594:0:80337:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5e8 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f78a0 expref: -99 pid: 80337 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571881.085616:0:80337:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5e8 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f78a0 expref: -99 pid: 80337 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571881.085624:0:80337:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5e8 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f78a0 expref: -99 pid: 80337 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571881.085632:0:80337:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5e8 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f78a0 expref: -99 pid: 80337 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788571881.085645:0:80337:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada60a49180 x1875449613604864/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788571897 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788571881.085662:0:80337:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5e8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f78a0 expref: -99 pid: 80337 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571881.085679:0:80337:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1788571881.085689:0:80337:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1788571881.085727:0:80337:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:0.0:1788571881.085735:0:80337:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d5e8 00000080:00010000:0.0:1788571881.085741:0:80337:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571881.085745:0:80337:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d5e8 from it ffffab0ec0d1fcb8 00010000:00010000:0.0:1788571881.085749:0:80337:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5e8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f78a0 expref: -99 pid: 80337 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571881.085764:0:80337:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5e8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f78a0 expref: -99 pid: 80337 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:0.0:1788571881.086771:0:80337:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1788571881.086781:0:80337:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571881.086806:0:80337:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571881.086824:0:80337:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5e8 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f78a0 expref: -99 pid: 80337 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571881.086838:0:80337:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5e8 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f78a0 expref: -99 pid: 80337 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571881.086846:0:80337:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788571881.086888:0:80337:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571881.086893:0:80337:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d5e8 00000080:00010000:0.0:1788571881.086899:0:80337:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d5e8 from it ffffab0ec0d1fcb8 00010000:00010000:0.0:1788571881.086903:0:80337:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5e8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f78a0 expref: -99 pid: 80337 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571881.086911:0:80337:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5e8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f78a0 expref: -99 pid: 80337 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788571881.087063:0:80337:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571881.087072:0:80337:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571881.087080:0:80337:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5e8 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f78a0 expref: -99 pid: 80337 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571881.087094:0:80337:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5e8 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f78a0 expref: -99 pid: 80337 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788571881.087100:0:80337:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788571881.087104:0:80337:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571881.087106:0:80337:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d5e8 00000080:00010000:0.0:1788571881.087110:0:80337:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d5e8 from it ffffab0ec0d1fbb0 00010000:00010000:0.0:1788571881.087113:0:80337:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5e8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f78a0 expref: -99 pid: 80337 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571881.087119:0:80337:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5e8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f78a0 expref: -99 pid: 80337 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1788571881.087128:0:80337:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:0.0:1788571881.087135:0:80337:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200080:0.0:1788571881.087234:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.087252:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00800000:00000004:0.0:1788571881.087261:0:80337:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: age 3761, max_age 3797 00000100:00100000:0.0:1788571881.087312:0:80337:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0e00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:80337:1875449613604992:192.168.202.113@tcp:41:lfs.0 00000100:00100000:0.0:1788571881.087412:0:80337:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada51952a80 going to sleep for 11 seconds 00000100:00100000:0.0:1788571881.092295:0:80337:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0e00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:80337:1875449613604992:192.168.202.113@tcp:41:lfs.0 00800000:00000004:0.0:1788571881.092336:0:80337:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: update ffff9ada4481d198 cache blocks 317364/353529 objects 1023691/1024000 00000080:00200080:0.0:1788571881.092446:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.092469:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00800000:00000004:0.0:1788571881.092478:0:80337:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: age 3761, max_age 3797 00000100:00100000:0.0:1788571881.092524:0:80337:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0e00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:80337:1875449613605120:192.168.202.113@tcp:41:lfs.0 00000100:00100000:0.0:1788571881.093352:0:80337:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada51952a80 going to sleep for 11 seconds 00000100:00100000:0.0:1788571881.097519:0:80337:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0e00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:80337:1875449613605120:192.168.202.113@tcp:41:lfs.0 00800000:00000004:0.0:1788571881.097562:0:80337:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: update ffff9ada44819e18 cache blocks 317408/353529 objects 1023738/1024000 00000080:00200080:0.0:1788571881.097674:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.097698:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.097707:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.097711:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.097716:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.097719:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.097723:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.097726:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.097730:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.097733:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.097737:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.097740:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.097744:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.097748:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.097752:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.097756:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.097759:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.097763:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.097766:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.097769:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.097775:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.097779:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.097783:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.097786:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.097791:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.097794:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.097799:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.097802:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.097806:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.097810:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.097814:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.097818:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.097822:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.097825:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.097830:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.097833:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.097837:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.097841:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.097846:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.097849:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.097914:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.097921:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.097926:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.097930:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.097935:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.097939:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.097943:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.097947:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.097951:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.097955:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.097958:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.097962:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.097965:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.097968:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.097972:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.097976:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.097990:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.097995:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.097999:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.098003:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.098006:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.098010:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.098013:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.098017:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.098021:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.098024:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.098028:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.098031:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.098034:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.098038:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.098043:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.098047:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.098051:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.098055:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.098059:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.098062:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.098067:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.098070:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.098074:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.098077:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.098082:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.098085:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.098090:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.098094:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.098100:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.098103:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.098107:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.098110:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.098113:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.098117:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.098121:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.098125:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.098129:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.098132:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.098137:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.098141:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.098145:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.098149:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.098153:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.098157:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.098160:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.098164:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.098168:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.098172:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.098175:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.098179:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.098182:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.098186:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.098189:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.098193:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.098197:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.098199:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.098203:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.098207:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.098211:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.098215:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.098219:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.098222:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.098226:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.098229:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.098234:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.098238:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.098241:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.098245:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.098249:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00800000:00000080:0.0:1788571881.098252:0:80337:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00000080:00200080:0.0:1788571881.098257:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00020000:00000080:0.0:1788571881.098264:0:80337:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00020000:00000004:0.0:1788571881.098275:0:80337:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: age 3761, max_age 3797 00000100:00100000:0.0:1788571881.098330:0:80337:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0e00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:80337:1875449613605248:192.168.202.113@tcp:13:lfs.0 00000100:00100000:0.0:1788571881.098452:0:80337:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada51952a80 going to sleep for 11 seconds 00000100:00100000:0.0:1788571881.101924:0:80337:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0e00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:80337:1875449613605248:192.168.202.113@tcp:13:lfs.0 00020000:00000004:0.0:1788571881.101966:0:80337:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: update ffff9ada4354d198 cache blocks 901357/958279 objects 261729/262144 00000080:00200080:0.0:1788571881.102073:0:80337:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc096fcf20 00020000:00000080:0.0:1788571881.102098:0:80337:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1000 uarg=0000000000000000 00020000:00000004:0.0:1788571881.102110:0:80337:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: age 3761, max_age 3797 00000100:00100000:0.0:1788571881.102159:0:80337:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0e00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:80337:1875449613605376:192.168.202.113@tcp:13:lfs.0 00000100:00100000:0.0:1788571881.102743:0:80337:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada51952a80 going to sleep for 11 seconds 00000100:00100000:0.0:1788571881.206834:0:15207:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f5500 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449613606016:192.168.202.113@tcp:503:ll_cfg_requeue.0 00000040:00080000:0.0:1788571881.255299:0:80357:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x2:0x0] index 64768 count 7 00000020:01000000:2.0:1788571881.255590:0:15207:0:(obd_config.c:2145:class_config_parse_llog()) Processed log params gen 7-6 (rc=0) 10000000:01000000:2.0:1788571881.255624:0:15207:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.202.113@tcp: configuration from log 'params' succeeded (0). 00010000:00010000:2.0:1788571881.255640:0:15207:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.202.113@tcp lock: ffff9ada53017e00/0xef30c340bfb3d5fd lrc: 3/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b82f78b5 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571881.255655:0:15207:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.202.113@tcp lock: ffff9ada53017e00/0xef30c340bfb3d5fd lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b82f78b5 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1788571881.255668:0:15207:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:2.0:1788571881.255671:0:15207:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9ada42e899c0 from 1 10000000:01000000:2.0:1788571881.255675:0:15207:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:2.0:1788571881.255733:0:15207:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.202.113@tcp lock: ffff9ada516e0600/0xef30c340bfb3d604 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571881.255747:0:15207:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.202.113@tcp lock: ffff9ada516e0600/0xef30c340bfb3d604 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571881.255761:0:15207:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.202.113@tcp lock: ffff9ada516e0600/0xef30c340bfb3d604 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571881.255803:0:15207:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35880 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449613606528:192.168.202.113@tcp:101:ll_cfg_requeue.0 00000100:00100000:2.0:1788571881.255947:0:15207:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6065fcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788571881.261272:0:15207:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a35880 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449613606528:192.168.202.113@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1788571881.261319:0:15207:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.202.113@tcp lock: ffff9ada516e0600/0xef30c340bfb3d604 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b82f78bc expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571881.261340:0:15207:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.202.113@tcp lock: ffff9ada516e0600/0xef30c340bfb3d604 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0xdb7edb52b82f78bc expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571881.261452:0:15207:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f3b80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449613606656:192.168.202.113@tcp:501:ll_cfg_requeue.0 00000100:00100000:0.0:1788571881.261548:0:15207:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada51952900 going to sleep for 11 seconds 00000100:00100000:0.0:1788571881.263584:0:15207:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f3b80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449613606656:192.168.202.113@tcp:501:ll_cfg_requeue.0 10000000:01000000:0.0:1788571881.263646:0:15207:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.202.113@tcp: configuration from log 'lustre-sptlrpc' failed (-2). 00010000:00010000:0.0:1788571881.263657:0:15207:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.202.113@tcp lock: ffff9ada516e0600/0xef30c340bfb3d604 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b82f78bc expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571881.263669:0:15207:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.202.113@tcp lock: ffff9ada516e0600/0xef30c340bfb3d604 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b82f78bc expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571882.359289:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788571883.392799:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200080:0.0:1788571883.538356:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.538381:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00800000:00000004:0.0:1788571883.538392:0:80392:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: age 3798, max_age 3800 00000100:00100000:0.0:1788571883.538450:0:80392:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f3b80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:80392:1875449613606784:192.168.202.113@tcp:41:lfs.0 00000100:00100000:0.0:1788571883.538565:0:80392:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada51952f00 going to sleep for 11 seconds 00000100:00100000:0.0:1788571883.541441:0:80392:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f3b80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:80392:1875449613606784:192.168.202.113@tcp:41:lfs.0 00800000:00000004:0.0:1788571883.541478:0:80392:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: update ffff9ada4481d198 cache blocks 317364/353529 objects 1023691/1024000 00000080:00200080:0.0:1788571883.541571:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.541592:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00800000:00000004:0.0:1788571883.541602:0:80392:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: age 3798, max_age 3800 00000100:00100000:0.0:1788571883.541801:0:80392:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f3b80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:80392:1875449613606912:192.168.202.113@tcp:41:lfs.0 00000100:00100000:0.0:1788571883.542360:0:80392:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada51952f00 going to sleep for 11 seconds 00000100:00100000:0.0:1788571883.552217:0:80392:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f3b80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:80392:1875449613606912:192.168.202.113@tcp:41:lfs.0 00800000:00000004:0.0:1788571883.552259:0:80392:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: update ffff9ada44819e18 cache blocks 317408/353529 objects 1023738/1024000 00000080:00200080:0.0:1788571883.552448:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552475:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552485:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552490:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552495:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552499:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552503:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552507:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552511:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552515:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552519:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552523:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552528:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552532:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552536:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552539:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552543:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552547:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552551:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552555:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552562:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552565:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552570:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552574:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552578:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552581:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552585:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552589:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552593:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552597:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552601:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552617:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552623:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552626:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552631:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552635:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552639:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552642:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552647:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552650:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552655:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552658:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552662:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552665:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552668:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552673:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552677:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552680:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552685:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552688:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552692:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552697:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552701:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552704:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552708:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552712:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552716:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552719:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552723:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552726:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552729:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552733:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552737:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552740:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552744:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552747:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552750:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552754:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552758:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552761:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552766:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552770:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552773:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552777:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552780:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552783:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552787:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552790:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552794:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552797:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552801:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552805:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552808:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552812:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552816:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552819:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552823:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552826:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552830:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552833:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552837:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552840:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552843:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552847:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552851:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552881:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552885:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552889:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552893:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552897:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552901:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552905:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552908:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552912:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552916:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552919:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552923:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552926:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552930:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552933:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552936:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552939:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552943:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552946:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552949:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552952:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552955:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552958:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552962:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552965:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552969:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552972:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552976:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.552979:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.552983:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00800000:00000080:0.0:1788571883.553000:0:80392:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571883.553007:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00020000:00000080:0.0:1788571883.553014:0:80392:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00020000:00000004:0.0:1788571883.553023:0:80392:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: age 3798, max_age 3800 00000100:00100000:0.0:1788571883.553076:0:80392:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f3b80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:80392:1875449613607040:192.168.202.113@tcp:13:lfs.0 00000100:00100000:0.0:1788571883.553194:0:80392:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada51952f00 going to sleep for 11 seconds 00000100:00100000:0.0:1788571883.559190:0:80392:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f3b80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:80392:1875449613607040:192.168.202.113@tcp:13:lfs.0 00020000:00000004:0.0:1788571883.559235:0:80392:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: update ffff9ada4354d198 cache blocks 901357/958279 objects 261729/262144 00000080:00200080:0.0:1788571883.559329:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00020000:00000080:0.0:1788571883.559352:0:80392:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00020000:00000004:0.0:1788571883.559362:0:80392:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: age 3798, max_age 3800 00000100:00100000:0.0:1788571883.559407:0:80392:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f3b80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:80392:1875449613607168:192.168.202.113@tcp:13:lfs.0 00000100:00100000:0.0:1788571883.559498:0:80392:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada51952f00 going to sleep for 11 seconds 00000080:00200080:3.0:1788571883.571351:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00020000:00000080:3.0:1788571883.571356:0:80392:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6ac00 uarg=0000000000000000 00000080:00200080:3.0:1788571883.571359:0:80392:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd9d01a7c0 00020000:00000080:3.0:1788571883.571363:0:80392:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6ac00 uarg=0000000000000000 00000080:00200008:3.0:1788571883.571382:0:80392:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00010000:00010000:3.0:1788571884.407268:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788571884.919604:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449613607296:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571884.919742:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613607552:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571884.919776:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613607808:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571884.919826:0:2383:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1788571884.919833:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613607424:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571884.919889:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613607680:192.168.202.113@tcp:400:kworker.0 00010000:00010000:2.0:1788571885.432558:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788571886.457695:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788571887.479607:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571888.505823:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788571889.528132:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788571890.039263:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449613607936:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571890.039546:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613608192:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571890.039582:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613608448:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571890.039617:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1788571890.039637:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613608064:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571890.039665:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613608320:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571890.049299:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613608064:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571890.049325:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449613607936:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571890.049339:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613608192:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571890.051549:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613608320:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571890.051571:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613608448:192.168.202.113@tcp:400:kworker.0 00010000:00010000:2.0:1788571891.575081:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788571892.599273:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:0.0:1788571893.025338:0:79931:0:(file.c:479:ll_file_release()) START file f34.replay-single:[0x200016b60:0xb:0x0](ffff9ada58217448), flags 100002 00000002:00100000:0.0:1788571893.025425:0:79931:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9ada516f1880 x1875449613597056/t197568495647(197568495647) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571858 ref 1 fl Complete:RMQU/604/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00000020:00100000:0.0:1788571893.025440:0:79931:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:0.0:1788571893.025447:0:79931:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada516f3800 x1875449613608704/t0(0) o35->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:0.0:1788571893.025473:0:79931:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f3800 pname:cluuid:pid:xid:nid:opc:job multiop:35eb2d60-2de0-4b00-9815-d5153edd0057:79931:1875449613608704:192.168.202.113@tcp:35:multiop.0 00000100:00100000:0.0:1788571893.025568:0:79931:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada519520c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788571893.030128:0:79931:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f3800 pname:cluuid:pid:xid:nid:opc:job multiop:35eb2d60-2de0-4b00-9815-d5153edd0057:79931:1875449613608704:192.168.202.113@tcp:35:multiop.0 00000002:00080000:0.0:1788571893.030151:0:79931:0:(mdc_request.c:1003:mdc_close()) @@@ Reset ESTALE = -116 req@ffff9ada516f3800 x1875449613608704/t0(0) o35->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:23/10 lens 392/456 e 0 to 0 dl 1788571909 ref 1 fl Complete:RQU/600/0 rc -116/-116 job:'multiop.0' uid:0 gid:0 projid:0 00000002:00100000:0.0:1788571893.030175:0:79931:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9ada516f1880 x1875449613597056/t197568495647(197568495647) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571858 ref 1 fl Complete:RMQU/604/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1788571893.030221:0:79931:0:(vvp_io.c:1826:vvp_io_init()) [0x200016b60:0xb:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1788571893.030233:0:79931:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1788571893.030262:0:79931:0:(vvp_io.c:304:vvp_io_fini()) [0x200016b60:0xb:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1788571893.030270:0:79931:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200016b60:0xb:0x0](ffff9ada58217448) 00000080:00200000:2.0:1788571893.042246:0:79791:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571893.042278:0:79791:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571893.042297:0:79791:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5e8 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f78a0 expref: -99 pid: 80337 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:2.0:1788571893.042311:0:79791:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5e8 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f78a0 expref: -99 pid: 80337 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000080:00200000:2.0:1788571893.042318:0:79791:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571893.042325:0:79791:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571893.042328:0:79791:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d5e8 00000080:00010000:2.0:1788571893.042335:0:79791:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d5e8 from it ffffab0ec0ddbb58 00010000:00010000:2.0:1788571893.042339:0:79791:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5e8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f78a0 expref: -99 pid: 80337 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:2.0:1788571893.042353:0:79791:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5e8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f78a0 expref: -99 pid: 80337 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00000080:00200000:2.0:1788571893.042366:0:79791:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:2.0:1788571893.042381:0:79791:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f34.replay-single, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), flags=9 00000080:00200000:2.0:1788571893.042385:0:79791:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f34.replay-single, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=getattr 00000002:00010000:2.0:1788571893.042397:0:79791:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f34.replay-single,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 01 00010000:00010000:2.0:1788571893.042443:0:79791:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1800/0xef30c340bfb3d60b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 79791 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571893.042452:0:79791:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1800/0xef30c340bfb3d60b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 79791 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571893.042463:0:79791:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1800/0xef30c340bfb3d60b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 79791 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571893.042490:0:79791:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35c00 pname:cluuid:pid:xid:nid:opc:job bash:35eb2d60-2de0-4b00-9815-d5153edd0057:79791:1875449613608832:192.168.202.113@tcp:101:bash.0 00000100:00100000:2.0:1788571893.042577:0:79791:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6065fd80 going to sleep for 11 seconds 00000100:00100000:3.0:1788571893.048483:0:79791:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a35c00 pname:cluuid:pid:xid:nid:opc:job bash:35eb2d60-2de0-4b00-9815-d5153edd0057:79791:1875449613608832:192.168.202.113@tcp:101:bash.0 00010000:00010000:3.0:1788571893.048506:0:79791:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1800/0xef30c340bfb3d60b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 79791 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788571893.048521:0:79791:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1800/0xef30c340bfb3d60b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 79791 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788571893.048528:0:79791:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1800/0xef30c340bfb3d60b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 79791 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788571893.048534:0:79791:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1800/0xef30c340bfb3d60b lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 79791 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788571893.048543:0:79791:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1800/0xef30c340bfb3d60b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 79791 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788571893.048548:0:79791:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada516e1800 already unused, calling callback (ffffffffc1966630) 00010000:00010000:3.0:1788571893.048553:0:79791:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1800/0xef30c340bfb3d60b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 79791 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788571893.048560:0:79791:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1800/0xef30c340bfb3d60b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 79791 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788571893.048567:0:79791:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1800/0xef30c340bfb3d60b lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 79791 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788571893.048574:0:79791:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1788571893.048581:0:79791:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1800/0xef30c340bfb3d60b lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 79791 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788571893.048587:0:79791:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1800/0xef30c340bfb3d60b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 79791 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1788571893.048605:0:79791:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9ada60a35c00 x1875449613608832/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/584 e 0 to 0 dl 1788571909 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'bash.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1788571893.048620:0:79791:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:3.0:1788571893.048647:0:79791:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5e8 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f78a0 expref: -99 pid: 80337 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00010000:00010000:3.0:1788571893.048660:0:79791:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5e8 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f78a0 expref: -99 pid: 80337 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00000080:00010000:3.0:1788571893.048669:0:79791:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d5e8 from it ffffab0ec0ddb930 00010000:00010000:3.0:1788571893.048673:0:79791:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5e8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f78a0 expref: -99 pid: 80337 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00010000:00010000:3.0:1788571893.048680:0:79791:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5e8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f78a0 expref: -99 pid: 80337 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:2.0:1788571894.648003:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788571895.167473:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613609088:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571895.167577:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613609344:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571895.167609:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:0.0:1788571895.167616:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449613608960:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571895.167644:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613609216:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571895.167667:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613609472:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571895.183741:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449613608960:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571895.183764:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613609216:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571895.183777:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613609472:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571895.183787:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613609088:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571895.183797:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613609344:192.168.202.113@tcp:400:kworker.0 00010000:00010000:3.0:1788571895.675041:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788571897.724876:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788571898.745310:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788571900.283638:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449613609600:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571900.283838:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613609728:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571900.283938:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613609856:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571900.283991:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613609984:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571900.284057:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613610112:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571900.288003:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613609984:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571900.288055:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613609728:192.168.202.113@tcp:400:kworker.0 00010000:00010000:3.0:1788571900.792035:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:0.0:1788571901.047844:0:81094:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == replay-single test 35: test recovery from llog for unlink op ========================================================== 21:31:39 (1788571899) 00000080:00200000:2.0:1788571901.095350:0:81096:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571901.095395:0:81096:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571901.095422:0:81096:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5e8 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f78a0 expref: -99 pid: 80337 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:2.0:1788571901.095444:0:81096:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5e8 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f78a0 expref: -99 pid: 80337 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00000080:00200000:2.0:1788571901.095460:0:81096:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571901.095470:0:81096:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571901.095475:0:81096:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d5e8 00000080:00010000:2.0:1788571901.095485:0:81096:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d5e8 from it ffffab0ec1c9fbf8 00010000:00010000:2.0:1788571901.095489:0:81096:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5e8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f78a0 expref: -99 pid: 80337 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:2.0:1788571901.095500:0:81096:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5e8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f78a0 expref: -99 pid: 80337 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00000080:00200000:2.0:1788571901.095518:0:81096:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:2.0:1788571901.095536:0:81096:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571901.095543:0:81096:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571901.095550:0:81096:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5e8 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f78a0 expref: -99 pid: 80337 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00010000:00010000:2.0:1788571901.095558:0:81096:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5e8 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f78a0 expref: -99 pid: 80337 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00000080:00200000:2.0:1788571901.095565:0:81096:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571901.095569:0:81096:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571901.095572:0:81096:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d5e8 00000080:00010000:2.0:1788571901.095575:0:81096:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d5e8 from it ffffab0ec1c9fb38 00010000:00010000:2.0:1788571901.095578:0:81096:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5e8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f78a0 expref: -99 pid: 80337 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00010000:00010000:2.0:1788571901.095590:0:81096:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763800/0xef30c340bfb3d5e8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f78a0 expref: -99 pid: 80337 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00000080:00200000:2.0:1788571901.095598:0:81096:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:2.0:1788571901.095603:0:81096:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f35.replay-single, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), file ffff9ada617ca300, open_flags 8941, mode 81b6 opened 0 00000080:00200000:2.0:1788571901.095613:0:81096:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f35.replay-single, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000100:00100000:2.0:1788571901.095690:0:81096:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37b80 pname:cluuid:pid:xid:nid:opc:job touch:35eb2d60-2de0-4b00-9815-d5153edd0057:81096:1875449613610240:192.168.202.113@tcp:700:touch.0 00000100:00100000:2.0:1788571901.095802:0:81096:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada51952540 going to sleep for 11 seconds 00000080:00200000:3.0:1788571901.181198:0:81096:0:(vvp_io.c:304:vvp_io_fini()) [0x200017331:0x1:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200008:3.0:1788571901.181230:0:81096:0:(llite_lib.c:2686:ll_setattr_raw()) COMPLETED file f35.replay-single:[0x200017331:0x1:0x0](ffff9ada58251148) current size 0, valid attrs 100201f0, mode 9ada, uid -1043726768, gid -21746, new size 18446744071751268887, atime 1788571901.000000000, mtime 1788571901.000000000, ctime 1788571901.000000000, rc 0 00000080:00200008:3.0:1788571901.181275:0:81096:0:(file.c:479:ll_file_release()) START file f35.replay-single:[0x200017331:0x1:0x0](ffff9ada58251148), flags 104001 00000002:00100000:3.0:1788571901.181378:0:81096:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9ada603f4e00 x1875449613610496/t201863462913(201863462913) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571917 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'touch.0' uid:0 gid:0 projid:0 00000020:00100000:3.0:1788571901.181392:0:81096:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:3.0:1788571901.181399:0:81096:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada60a49880 x1875449613611264/t0(0) o35->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:3.0:1788571901.181432:0:81096:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a49880 pname:cluuid:pid:xid:nid:opc:job touch:35eb2d60-2de0-4b00-9815-d5153edd0057:81096:1875449613611264:192.168.202.113@tcp:35:touch.0 00000100:00100000:3.0:1788571901.181550:0:81096:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada429ad6c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788571901.187108:0:81096:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a49880 pname:cluuid:pid:xid:nid:opc:job touch:35eb2d60-2de0-4b00-9815-d5153edd0057:81096:1875449613611264:192.168.202.113@tcp:35:touch.0 00000002:00100000:0.0:1788571901.187165:0:81096:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9ada603f4e00 x1875449613610496/t201863462913(201863462913) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571917 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'touch.0' uid:0 gid:0 projid:0 00010000:00010000:3.0:1788571901.815556:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:2.0:1788571902.138187:0:81115:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571902.138574:0:81115:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571902.138664:0:81115:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d620 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 81115 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571902.138676:0:81115:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d620 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 81115 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571902.138689:0:81115:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d620 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 81115 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571902.138724:0:81115:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35180 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:81115:1875449613611392:192.168.202.113@tcp:101:rm.0 00000100:00100000:2.0:1788571902.138843:0:81115:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada51952f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788571902.145801:0:81115:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a35180 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:81115:1875449613611392:192.168.202.113@tcp:101:rm.0 00010000:00010000:2.0:1788571902.145841:0:81115:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d620 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7910 expref: -99 pid: 81115 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571902.146484:0:81115:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d620 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7910 expref: -99 pid: 81115 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571902.146505:0:81115:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d620 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7910 expref: -99 pid: 81115 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571902.146519:0:81115:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d620 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7910 expref: -99 pid: 81115 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571902.146539:0:81115:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d620 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7910 expref: -99 pid: 81115 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571902.146549:0:81115:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d620 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7910 expref: -99 pid: 81115 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571902.146557:0:81115:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d620 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7910 expref: -99 pid: 81115 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1788571902.146574:0:81115:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada60a35180 x1875449613611392/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788571918 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1788571902.146593:0:81115:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d620 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7910 expref: -99 pid: 81115 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571902.146611:0:81115:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:2.0:1788571902.146620:0:81115:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:2.0:1788571902.146738:0:81115:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:2.0:1788571902.146747:0:81115:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d620 00000080:00010000:2.0:1788571902.146754:0:81115:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571902.146758:0:81115:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d620 from it ffffab0ec0c87b58 00010000:00010000:2.0:1788571902.146762:0:81115:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d620 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7910 expref: -99 pid: 81115 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571902.146774:0:81115:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d620 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7910 expref: -99 pid: 81115 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571902.146812:0:81115:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:2.0:1788571902.146837:0:81115:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f35.replay-single, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), flags=8 00000080:00200000:2.0:1788571902.146843:0:81115:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f35.replay-single, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=getattr 00000002:00010000:2.0:1788571902.147065:0:81115:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f35.replay-single,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 01 00010000:00010000:2.0:1788571902.147123:0:81115:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0800/0xef30c340bfb3d627 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 81115 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571902.147132:0:81115:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0800/0xef30c340bfb3d627 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 81115 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571902.147143:0:81115:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0800/0xef30c340bfb3d627 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 81115 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571902.147173:0:81115:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35180 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:81115:1875449613611520:192.168.202.113@tcp:101:rm.0 00000100:00100000:2.0:1788571902.147373:0:81115:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada51952f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788571902.156113:0:81115:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a35180 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:81115:1875449613611520:192.168.202.113@tcp:101:rm.0 00010000:00010000:2.0:1788571902.156155:0:81115:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0800/0xef30c340bfb3d627 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f792c expref: -99 pid: 81115 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571902.156179:0:81115:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0800/0xef30c340bfb3d627 lrc: 4/1,0 mode: --/PR res: [0x200017331:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f792c expref: -99 pid: 81115 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571902.156189:0:81115:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0800/0xef30c340bfb3d627 lrc: 4/1,0 mode: --/PR res: [0x200017331:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f792c expref: -99 pid: 81115 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571902.156206:0:81115:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0800/0xef30c340bfb3d627 lrc: 4/1,0 mode: PR/PR res: [0x200017331:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f792c expref: -99 pid: 81115 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571902.156219:0:81115:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0800/0xef30c340bfb3d627 lrc: 4/1,0 mode: PR/PR res: [0x200017331:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f792c expref: -99 pid: 81115 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571902.156236:0:81115:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0800/0xef30c340bfb3d627 lrc: 5/2,0 mode: PR/PR res: [0x200017331:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f792c expref: -99 pid: 81115 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571902.156246:0:81115:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0800/0xef30c340bfb3d627 lrc: 5/2,0 mode: PR/PR res: [0x200017331:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f792c expref: -99 pid: 81115 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571902.156254:0:81115:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0800/0xef30c340bfb3d627 lrc: 4/1,0 mode: PR/PR res: [0x200017331:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f792c expref: -99 pid: 81115 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1788571902.156268:0:81115:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada60a35180 x1875449613611520/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/640 e 0 to 0 dl 1788571918 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1788571902.156280:0:81115:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: getattr, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0800/0xef30c340bfb3d627 lrc: 3/1,0 mode: PR/PR res: [0x200017331:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f792c expref: -99 pid: 81115 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1788571902.156295:0:81115:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0800/0xef30c340bfb3d627 lrc: 3/1,0 mode: PR/PR res: [0x200017331:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f792c expref: -99 pid: 81115 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571902.156306:0:81115:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8590029617/1 (0 0) 00000080:00200000:2.0:1788571902.156313:0:81115:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1788571902.156381:0:81115:0:(namei.c:147:ll_iget()) got inode: [0x200017331:0x1:0x0](ffff9ada58251148): rc = 0 00000080:00200000:2.0:1788571902.156401:0:81115:0:(file.c:6843:ll_layout_conf()) [0x200017331:0x1:0x0]: layout version change: 0 -> 0 00000080:00010000:2.0:1788571902.156407:0:81115:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200017331:0x1:0x0](ffff9ada58251148) for lock 0xef30c340bfb3d627 00000080:00010000:2.0:1788571902.156422:0:81115:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200017331:0x1:0x0](ffff9ada58251148) 00000080:00010000:2.0:1788571902.156427:0:81115:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d627 from it ffffab0ec0c87b20 00010000:00010000:2.0:1788571902.156431:0:81115:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0800/0xef30c340bfb3d627 lrc: 3/1,0 mode: PR/PR res: [0x200017331:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f792c expref: -99 pid: 81115 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571902.156442:0:81115:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0800/0xef30c340bfb3d627 lrc: 2/0,0 mode: PR/PR res: [0x200017331:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f792c expref: -99 pid: 81115 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200008:2.0:1788571902.156494:0:81115:0:(file.c:6296:ll_getattr_dentry()) START file f35.replay-single:[0x200017331:0x1:0x0](ffff9ada58251148), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1788571902.156502:0:81115:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200017331:0x1:0x0](ffff9ada58251148),name=f35.replay-single 00000002:00010000:2.0:1788571902.156531:0:81115:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200017331:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571902.156541:0:81115:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0800/0xef30c340bfb3d627 lrc: 2/1,0 mode: PR/PR res: [0x200017331:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f792c expref: -99 pid: 81115 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571902.156553:0:81115:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0800/0xef30c340bfb3d627 lrc: 2/1,0 mode: PR/PR res: [0x200017331:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f792c expref: -99 pid: 81115 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571902.156561:0:81115:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571902.156565:0:81115:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200017331:0x1:0x0](ffff9ada58251148) 00000080:00010000:2.0:1788571902.156569:0:81115:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200017331:0x1:0x0](ffff9ada58251148) for lock 0xef30c340bfb3d627 00000080:00010000:2.0:1788571902.156573:0:81115:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d627 from it ffffab0ec0c87cb8 00010000:00010000:2.0:1788571902.156576:0:81115:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0800/0xef30c340bfb3d627 lrc: 3/1,0 mode: PR/PR res: [0x200017331:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f792c expref: -99 pid: 81115 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571902.156584:0:81115:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0800/0xef30c340bfb3d627 lrc: 2/0,0 mode: PR/PR res: [0x200017331:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f792c expref: -99 pid: 81115 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571902.156611:0:81115:0:(vvp_io.c:1826:vvp_io_init()) [0x200017331:0x1:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:2.0:1788571902.156624:0:81115:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x200017331:0x1:0x0] 00000008:00010000:2.0:1788571902.156646:0:81115:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: a 00000008:00010000:2.0:1788571902.156650:0:81115:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff9ada5163a730, osc lock ffff9ada4880eab0, flags 0x400001002 00010000:00010000:2.0:1788571902.156698:0:81115:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 0000000099f0bd80/0xef30c340bfb3d62e lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x622:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 81115 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571902.156709:0:81115:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x400001000 ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 0000000099f0bd80/0xef30c340bfb3d62e lrc: 3/1,0 mode: --/PR res: [0x2c0000401:0x622:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 81115 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788571902.156803:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613611648:192.168.202.113@tcp:101:rm.0 00000020:00200000:2.0:1788571902.157049:0:81115:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00010000:00010000:2.0:1788571902.166593:0:2386:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 0000000099f0bd80/0xef30c340bfb3d62e lrc: 6/2,0 mode: --/PR res: [0x2c0000401:0x622:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 81115 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571902.166655:0:2386:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 0000000099f0bd80/0xef30c340bfb3d62e lrc: 6/2,0 mode: --/PR res: [0x2c0000401:0x622:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7933 expref: -99 pid: 81115 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571902.166674:0:2386:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 0000000099f0bd80/0xef30c340bfb3d62e lrc: 6/2,0 mode: PR/PR res: [0x2c0000401:0x622:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7933 expref: -99 pid: 81115 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571902.166692:0:2386:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 0000000099f0bd80/0xef30c340bfb3d62e lrc: 6/2,0 mode: PR/PR res: [0x2c0000401:0x622:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7933 expref: -99 pid: 81115 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:2.0:1788571902.166709:0:2386:0:(osc_lock.c:187:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 0000000099f0bd80/0xef30c340bfb3d62e lrc: 5/2,0 mode: PR/PR res: [0x2c0000401:0x622:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f7933 expref: -99 pid: 81115 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000020:00200000:2.0:1788571902.166730:0:2386:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00010000:00010000:2.0:1788571902.166742:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 0000000099f0bd80/0xef30c340bfb3d62e lrc: 6/2,0 mode: PR/PR res: [0x2c0000401:0x622:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0xdb7edb52b82f7933 expref: -99 pid: 81115 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571902.166751:0:2386:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 0000000099f0bd80/0xef30c340bfb3d62e lrc: 5/1,0 mode: PR/PR res: [0x2c0000401:0x622:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0xdb7edb52b82f7933 expref: -99 pid: 81115 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571902.166759:0:2386:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 0000000099f0bd80/0xef30c340bfb3d62e lrc: 5/1,0 mode: PR/PR res: [0x2c0000401:0x622:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0xdb7edb52b82f7933 expref: -99 pid: 81115 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571902.166768:0:2386:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 0000000099f0bd80/0xef30c340bfb3d62e lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x622:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0xdb7edb52b82f7933 expref: -99 pid: 81115 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000100:00100000:2.0:1788571902.166797:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a35180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613611648:192.168.202.113@tcp:101:rm.0 00000080:00200000:0.0:1788571902.167207:0:81115:0:(file.c:1775:ll_merge_attr_nolock()) [0x200017331:0x1:0x0] updating i_size 0 i_blocks 0 00000020:00010000:0.0:1788571902.167214:0:81115:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9ada60bc39d0 (ffff9ada60b0d248/0) at cl_lock_release():245 00000020:00010000:0.0:1788571902.167218:0:81115:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9ada60bc39d0 (ffff9ada60b0d248/0) at cl_lock_cancel():142 00000020:00010000:0.0:1788571902.167225:0:81115:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9ada5163a730 (ffff9ada60b0d098/1) at cl_lock_cancel():142 00000020:00010000:0.0:1788571902.167244:0:81115:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9ada60bc39d0 (ffff9ada60b0d248/0) at cl_lock_fini():72 00000020:00010000:0.0:1788571902.167246:0:81115:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9ada5163a730 (ffff9ada60b0d248/1) at cl_lock_fini():72 00000080:00200000:0.0:1788571902.167264:0:81115:0:(vvp_io.c:304:vvp_io_fini()) [0x200017331:0x1:0x0] ignore/verify layout 0/1, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1788571902.167343:0:81115:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571902.167375:0:81115:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571902.167400:0:81115:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d620 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7910 expref: -99 pid: 81115 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571902.167415:0:81115:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d620 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7910 expref: -99 pid: 81115 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571902.167424:0:81115:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788571902.167433:0:81115:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571902.167437:0:81115:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d620 00000080:00010000:0.0:1788571902.167446:0:81115:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d620 from it ffffab0ec0c87c10 00010000:00010000:0.0:1788571902.167449:0:81115:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d620 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7910 expref: -99 pid: 81115 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571902.167458:0:81115:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d620 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7910 expref: -99 pid: 81115 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788571902.167470:0:81115:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:0.0:1788571902.167483:0:81115:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f35.replay-single, flags=0 00000080:00200000:0.0:1788571902.167490:0:81115:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571902.167497:0:81115:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571902.167503:0:81115:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d620 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7910 expref: -99 pid: 81115 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571902.167516:0:81115:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d620 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7910 expref: -99 pid: 81115 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788571902.167523:0:81115:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788571902.167527:0:81115:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571902.167529:0:81115:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d620 00000080:00010000:0.0:1788571902.167533:0:81115:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d620 from it ffffab0ec0c87d68 00010000:00010000:0.0:1788571902.167535:0:81115:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d620 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7910 expref: -99 pid: 81115 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571902.167543:0:81115:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d620 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7910 expref: -99 pid: 81115 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1788571902.167551:0:81115:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:0.0:1788571902.167557:0:81115:0:(namei.c:2351:ll_unlink()) VFS Op:name=f35.replay-single, dir=[0x200000007:0x1:0x0](ffff9ada58125b88) 00010000:00010000:0.0:1788571902.167591:0:81115:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d620 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b82f7910 expref: -99 pid: 81115 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1788571902.167619:0:81115:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0800/0xef30c340bfb3d627 lrc: 2/0,0 mode: PR/PR res: [0x200017331:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0xdb7edb52b82f792c expref: -99 pid: 81115 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788571902.167631:0:81115:0:(vvp_object.c:117:vvp_conf_set()) [0x200017331:0x1:0x0]: losing layout lock 00010000:00010000:0.0:1788571902.167643:0:81115:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016800/0xef30c340bfb3d619 lrc: 2/0,0 mode: CR/CR res: [0x200017331:0x1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0xdb7edb52b82f7902 expref: -99 pid: 81096 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788571902.167665:0:81115:0:(vvp_object.c:117:vvp_conf_set()) [0x200017331:0x1:0x0]: losing layout lock 00010000:00010000:0.0:1788571902.167704:0:81115:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0800/0xef30c340bfb3d627 lrc: 1/0,0 mode: --/PR res: [0x200017331:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0xdb7edb52b82f792c expref: -99 pid: 81115 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571902.167712:0:81115:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016800/0xef30c340bfb3d619 lrc: 1/0,0 mode: --/CR res: [0x200017331:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0xdb7edb52b82f7902 expref: -99 pid: 81096 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571902.167719:0:81115:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d620 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f7910 expref: -99 pid: 81115 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1788571902.167725:0:81115:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 3 locks packed 00010000:00010000:0.0:1788571902.167728:0:81115:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0800/0xef30c340bfb3d627 lrc: 0/0,0 mode: --/PR res: [0x200017331:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0xdb7edb52b82f792c expref: -99 pid: 81115 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571902.167745:0:81115:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53016800/0xef30c340bfb3d619 lrc: 0/0,0 mode: --/CR res: [0x200017331:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0xdb7edb52b82f7902 expref: -99 pid: 81096 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571902.167761:0:81115:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb3d620 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f7910 expref: -99 pid: 81115 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:0.0:1788571902.167782:0:81115:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1788571902.167794:0:81115:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada516f1880 x1875449613611904/t0(0) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 512/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788571902.167824:0:81115:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f1880 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:81115:1875449613611904:192.168.202.113@tcp:36:rm.0 00000100:00100000:0.0:1788571902.168383:0:81115:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada46424480 going to sleep for 11 seconds 00000080:00200000:3.0:1788571903.145749:0:48662:0:(vvp_io.c:1826:vvp_io_init()) [0x200017331:0x1:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1788571903.145769:0:48662:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1788571903.145807:0:48662:0:(vvp_io.c:304:vvp_io_fini()) [0x200017331:0x1:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00010000:00010000:3.0:1788571903.863638:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788571904.889082:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788571905.404574:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449613612032:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571905.404687:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613612288:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571905.404717:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613612544:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571905.404752:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1788571905.404759:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613612160:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571905.404780:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613612416:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571905.417258:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613612160:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571905.417281:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613612416:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571905.417294:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449613612032:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571905.417306:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613612288:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571905.417317:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613612544:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788571906.424228:0:2377:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.113@tcp (source: 12345-192.168.202.113@tcp) 00000100:00100000:3.0:1788571906.424282:0:33411:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875449638561536 00000100:00100000:3.0:1788571906.424306:0:33411:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875449638561536 00000100:00100000:3.0:1788571906.424375:0:33411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.113@tcp, seq: 236 00000100:00100000:3.0:1788571906.424388:0:33411:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9ada60a48e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_003:LOV_OSC_UUID+4:43970:x1875449638561536:12345-192.168.202.113@tcp:104: 00010000:00010000:3.0:1788571906.424414:0:33411:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 0000000099f0bd80/0xef30c340bfb3d62e lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x622:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000010000 nid: local remote: 0xdb7edb52b82f7933 expref: -99 pid: 81115 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1788571906.425031:0:33411:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1788571906.425051:0:33411:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9ada60a48e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_003:LOV_OSC_UUID+4:43970:x1875449638561536:12345-192.168.202.113@tcp:104: Request processed in 665us (839us total) trans 0 rc 0/0 00000100:00100000:3.0:1788571906.425061:0:33411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.113@tcp, seq: 236 00010000:00010000:0.0:1788571906.425263:0:15205:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1788571906.425273:0:15205:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 0000000099f0bd80/0xef30c340bfb3d62e lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x622:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800420000010000 nid: local remote: 0xdb7edb52b82f7933 expref: -99 pid: 81115 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571906.425294:0:15205:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada516e0e00 already unused, calling callback (ffffffffc102e300) 00010000:00010000:0.0:1788571906.425302:0:15205:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 0000000099f0bd80/0xef30c340bfb3d62e lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x622:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800428400010000 nid: local remote: 0xdb7edb52b82f7933 expref: -99 pid: 81115 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571906.425328:0:15205:0:(vvp_io.c:1826:vvp_io_init()) [0x200017331:0x1:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:0.0:1788571906.425344:0:15205:0:(vvp_io.c:304:vvp_io_fini()) [0x200017331:0x1:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:0.0:1788571906.425401:0:15205:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 0000000099f0bd80/0xef30c340bfb3d62e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x622:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400010020 nid: local remote: 0xdb7edb52b82f7933 expref: -99 pid: 81115 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571906.425417:0:15205:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1788571906.425452:0:15205:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 0000000099f0bd80/0xef30c340bfb3d62e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x622:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400010020 nid: local remote: 0xdb7edb52b82f7933 expref: -99 pid: 81115 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571906.425462:0:15205:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 0000000099f0bd80/0xef30c340bfb3d62e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x622:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804c69400010020 nid: local remote: 0xdb7edb52b82f7933 expref: -99 pid: 81115 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571906.426301:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788571906.426305:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788571906.426307:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788571906.939003:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788571908.984425:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788571910.010602:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788571910.531701:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449613612800:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571910.531827:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613613056:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571910.531877:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a36680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613613312:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571910.531913:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1788571910.531919:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613612928:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571910.531939:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a36300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613613184:192.168.202.113@tcp:400:kworker.0 00000100:00080000:3.0:1788571910.551187:0:2385:0:(recover.c:205:ptlrpc_request_handle_notconn()) import lustre-MDT0000-mdc-ffff9ada58b0a000 of lustre-MDT0000_UUID@192.168.202.113@tcp abruptly disconnected: reconnecting 00000100:00080000:3.0:1788571910.551200:0:2385:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:3.0:1788571910.551207:0:2385:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff9ada58b0a000: Connection to lustre-MDT0000 (at 192.168.202.113@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:3.0:1788571910.551216:0:2385:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788571910.551222:0:2385:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 3792 00000100:00080000:3.0:1788571910.551237:0:2385:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788571910.551293:0:2385:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788571910.551331:0:2385:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60a49c00 x1875449613613440/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788571910.551365:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a35500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613612928:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788571910.551391:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a36300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613613184:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788571910.551406:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613613056:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788571910.551418:0:2385:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-ffff9ada58b0a000: committing for last_committed 4294967635 gen 5 00000100:00100000:3.0:1788571910.551438:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a36680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613613312:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788571910.551472:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a49c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449613613440:192.168.202.113@tcp:38:ptlrpcd_01_00.0 00000100:00080000:3.0:1788571910.565817:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788571910.565840:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-19) 00000100:00100000:3.0:1788571910.565846:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a49c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449613613440:192.168.202.113@tcp:38:ptlrpcd_01_00.0 00010000:00010000:0.0:1788571912.056571:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788571913.084448:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788571914.619518:0:81115:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada46424480 going to sleep for 1 seconds 00010000:00010000:0.0:1788571915.128175:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788571915.639022:0:81115:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada46424480 going to sleep for 1 seconds 00000100:00100000:2.0:1788571915.644783:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a49c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449613613568:192.168.202.113@tcp:400:kworker.0 00000100:00080000:3.0:1788571915.645009:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788571915.645015:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788571915.645020:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788571915.645028:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 3828 00000100:00080000:3.0:1788571915.645044:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 5s 00000100:00080000:3.0:1788571915.645054:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788571915.645068:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788571915.645094:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60a48000 x1875449613613696/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788571915.653813:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1788571915.653866:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a4a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613613824:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571915.658174:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a4ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613614080:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571915.658345:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1788571915.658354:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a4b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613613952:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788571915.670171:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613613824:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788571915.670723:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613613952:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788571915.670740:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613614080:192.168.202.113@tcp:400:kworker.0 00010000:00010000:3.0:1788571916.162471:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788571916.663191:0:81115:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada46424480 going to sleep for 1 seconds 00000100:00100000:0.0:1788571917.687152:0:81115:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada46424480 going to sleep for 1 seconds 00010000:00010000:3.0:1788571918.206724:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000400:0.0:1788571918.714055:0:81115:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788571902/real 1788571902] req@ffff9ada516f1880 x1875449613611904/t0(0) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 512/624 e 0 to 1 dl 1788571918 ref 2 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788571918.714106:0:81115:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff9ada51140000 already not connected (conn 322, was 320): DISCONN 00000100:00100000:0.0:1788571918.714114:0:81115:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada46424480 going to sleep for 0 seconds 00010000:00010000:3.0:1788571919.226487:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788571919.735068:0:81115:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada46424480 going to sleep for 0 seconds 00000100:00100000:0.0:1788571920.759387:0:81115:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada46424480 going to sleep for 0 seconds 00000100:00100000:3.0:1788571920.763558:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449613614336:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788571920.770543:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449613614848:192.168.202.113@tcp:250:ptlrpcd_00_01.0 00000100:00100000:3.0:1788571920.770575:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a35180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449613612800:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571920.770723:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a49c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449613613568:192.168.202.113@tcp:400:kworker.0 00000100:00080000:0.0:1788571920.785890:0:2382:0:(import.c:1176:ptlrpc_connect_interpret()) MGC192.168.202.113@tcp: connect to target with instance 0 10000000:01000000:0.0:1788571920.785920:0:2382:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808005 00000100:02000400:0.0:1788571920.785933:0:2382:0:(import.c:1328:ptlrpc_connect_interpret()) Evicted from MGS (at 192.168.202.113@tcp) after server handle changed from 0xdb7edb52b82f7892 to 0xdb7edb52b82f7d1c 00000100:00080000:0.0:1788571920.785939:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to EVICTED 00000100:00080000:0.0:1788571920.785948:0:2382:0:(import.c:1718:ptlrpc_import_recovery_state_machine()) evicted from MGS@192.168.202.113@tcp; invalidating 00000100:00100000:0.0:1788571920.786403:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449613614848:192.168.202.113@tcp:250:ptlrpcd_00_01.0 00010000:00080000:0.0:1788571920.786437:0:2382:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff9ada603f4000 x1875449613614336/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/416 e 0 to 0 dl 1788571940 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788571920.786454:0:2382:0:(import.c:1176:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to target with instance 48 00000100:00080000:0.0:1788571920.786471:0:2382:0:(import.c:1382:ptlrpc_connect_interpret()) @@@ lustre-MDT0000-mdc-ffff9ada58b0a000: evicting (reconnect/recover flags not set: 4) req@ffff9ada603f4000 x1875449613614336/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/416 e 0 to 0 dl 1788571940 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788571920.786480:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to EVICTED 00000100:02020000:0.0:1788571920.786484:0:2382:0:(import.c:1707:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff9ada58b0a000: This client was evicted by lustre-MDT0000; in progress operations using this service will fail. 00000100:00080000:2.0:1788571920.786672:0:81345:0:(import.c:1642:ptlrpc_invalidate_import_thread()) thread invalidate import MGC192.168.202.113@tcp to MGS@192.168.202.113@tcp 00000100:00100000:2.0:1788571920.786677:0:81345:0:(import.c:323:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 10000000:01000000:2.0:1788571920.786682:0:81345:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808003 00010000:00010000:2.0:1788571920.786700:0:81345:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.202.113@tcp lock: ffff9ada51763a00/0xef30c340bfb3d5f6 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x110e400000000 nid: local remote: 0xdb7edb52b82f78ae expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1788571920.786712:0:81345:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.202.113@tcp lock: ffff9ada51763a00/0xef30c340bfb3d5f6 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x110f400000000 nid: local remote: 0xdb7edb52b82f78ae expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1788571920.786719:0:81345:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1788571920.786742:0:81345:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571920.786753:0:81345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.202.113@tcp lock: ffff9ada51763a00/0xef30c340bfb3d5f6 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x590f400000000 nid: local remote: 0xdb7edb52b82f78ae expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571920.786767:0:81345:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.202.113@tcp lock: ffff9ada516e0600/0xef30c340bfb3d604 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0xdb7edb52b82f78bc expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1788571920.786774:0:81345:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.202.113@tcp lock: ffff9ada516e0600/0xef30c340bfb3d604 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0xdb7edb52b82f78bc expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1788571920.786780:0:81345:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1788571920.786784:0:81345:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571920.786787:0:81345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.202.113@tcp lock: ffff9ada516e0600/0xef30c340bfb3d604 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0xdb7edb52b82f78bc expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571920.786802:0:81345:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.202.113@tcp lock: ffff9ada51762e00/0xef30c340bfb3d5ef lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0xdb7edb52b82f78a7 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1788571920.786808:0:81345:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.202.113@tcp lock: ffff9ada51762e00/0xef30c340bfb3d5ef lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0xdb7edb52b82f78a7 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1788571920.786814:0:81345:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:2.0:1788571920.786817:0:81345:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571920.786820:0:81345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.202.113@tcp lock: ffff9ada51762e00/0xef30c340bfb3d5ef lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0xdb7edb52b82f78a7 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571920.786838:0:81345:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.202.113@tcp lock: ffff9ada53017e00/0xef30c340bfb3d5fd lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0xdb7edb52b82f78b5 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1788571920.786845:0:81345:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.202.113@tcp lock: ffff9ada53017e00/0xef30c340bfb3d5fd lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0xdb7edb52b82f78b5 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1788571920.786850:0:81345:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:2.0:1788571920.786899:0:81345:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571920.786902:0:81345:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.202.113@tcp lock: ffff9ada53017e00/0xef30c340bfb3d5fd lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0xdb7edb52b82f78b5 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1788571920.786918:0:81345:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from EVICTED to RECOVER 00000100:00080000:2.0:1788571920.786922:0:81345:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from RECOVER to FULL 10000000:01000000:2.0:1788571920.786924:0:81345:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808004 00000100:00080000:2.0:1788571920.786928:0:81345:0:(pinger.c:186:ptlrpc_pinger_ir_up()) IR up 00000100:02000000:2.0:1788571920.786931:0:81345:0:(import.c:1787:ptlrpc_import_recovery_state_machine()) MGC192.168.202.113@tcp: Connection restored to 192.168.202.113@tcp (at 192.168.202.113@tcp) 00000100:00080000:0.0:1788571920.805851:0:2382:0:(import.c:1718:ptlrpc_import_recovery_state_machine()) evicted from lustre-MDT0000_UUID@192.168.202.113@tcp; invalidating 00000100:00100000:0.0:1788571920.807400:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449613614336:192.168.202.113@tcp:38:kworker.0 00000100:00080000:2.0:1788571920.809074:0:81346:0:(import.c:1642:ptlrpc_invalidate_import_thread()) thread invalidate import lustre-MDT0000-mdc-ffff9ada58b0a000 to lustre-MDT0000_UUID@192.168.202.113@tcp 00000100:00080000:2.0:1788571920.809083:0:81346:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import lustre-MDT0000_UUID INVALID 00000100:00100000:2.0:1788571920.809098:0:81346:0:(client.c:3603:ptlrpc_abort_inflight()) @@@ aborting waiting req req@ffff9ada516f1880 x1875449613611904/t0(0) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 512/624 e 0 to 1 dl 1788571918 ref 2 fl Rpc:XQU/200/ffffffff rc 0/-1 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788571920.809122:0:81346:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 197568495648 gen 8 00000100:00100000:2.0:1788571920.809130:0:81346:0:(client.c:3164:ptlrpc_free_committed()) @@@ free request with old gen req@ffff9ada603f4e00 x1875449613610496/t201863462913(201863462913) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788571917 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'touch.0' uid:0 gid:0 projid:0 00000100:00100000:2.0:1788571920.809160:0:81346:0:(client.c:3164:ptlrpc_free_committed()) @@@ free request with old gen req@ffff9ada603f5880 x1875449613610752/t201863462914(201863462914) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 488/456 e 0 to 0 dl 1788571917 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'touch.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788571920.809174:0:81346:0:(client.c:3164:ptlrpc_free_committed()) @@@ free request with old gen req@ffff9ada60a49880 x1875449613611264/t201863462915(201863462915) o35->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:23/10 lens 392/456 e 0 to 0 dl 1788571917 ref 1 fl Complete:RQU/604/0 rc 0/0 job:'touch.0' uid:0 gid:0 projid:0 00000100:00100000:2.0:1788571920.809205:0:81346:0:(import.c:323:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 00000100:00100000:0.0:1788571920.809470:0:81115:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f1880 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:81115:1875449613611904:192.168.202.113@tcp:36:rm.0 00000100:00080000:2.0:1788571920.819786:0:81346:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from EVICTED to RECOVER 00000100:00080000:2.0:1788571920.819803:0:81346:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from RECOVER to FULL 00000100:02000000:3.0:1788571920.824067:0:81346:0:(import.c:1787:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff9ada58b0a000: Connection restored to 192.168.202.113@tcp (at 192.168.202.113@tcp) 00010000:00010000:3.0:1788571921.271529:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788571922.295244:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:01000000:0.0:1788571922.446659:0:15207:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 52-51 (rc=0) 10000000:01000000:0.0:1788571922.446679:0:15207:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.202.113@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:0.0:1788571922.446695:0:15207:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.202.113@tcp lock: ffff9ada53016800/0xef30c340bfb3d63c lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b82f7d2a expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571922.446709:0:15207:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.202.113@tcp lock: ffff9ada53016800/0xef30c340bfb3d63c lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b82f7d2a expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1788571922.446721:0:15207:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:0.0:1788571922.446724:0:15207:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9ada58b0a000 from 7 10000000:01000000:0.0:1788571922.446728:0:15207:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:0.0:1788571922.446790:0:15207:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.202.113@tcp lock: ffff9ada51617a00/0xef30c340bfb3d643 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571922.446800:0:15207:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.202.113@tcp lock: ffff9ada51617a00/0xef30c340bfb3d643 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571922.446813:0:15207:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.202.113@tcp lock: ffff9ada51617a00/0xef30c340bfb3d643 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571922.446850:0:15207:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f1880 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449613615616:192.168.202.113@tcp:101:ll_cfg_requeue.0 00000100:00100000:0.0:1788571922.447039:0:15207:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada46424480 going to sleep for 11 seconds 00000100:00100000:0.0:1788571922.454794:0:15207:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f1880 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449613615616:192.168.202.113@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1788571922.454843:0:15207:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.202.113@tcp lock: ffff9ada51617a00/0xef30c340bfb3d643 lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b82f7d31 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571922.454894:0:15207:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.202.113@tcp lock: ffff9ada51617a00/0xef30c340bfb3d643 lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0xdb7edb52b82f7d31 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571922.455022:0:15207:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f1880 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449613615744:192.168.202.113@tcp:501:ll_cfg_requeue.0 00000100:00100000:0.0:1788571922.455127:0:15207:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada46424480 going to sleep for 11 seconds 00000100:00100000:0.0:1788571922.468494:0:15207:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f1880 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449613615744:192.168.202.113@tcp:501:ll_cfg_requeue.0 00000100:00100000:0.0:1788571922.468579:0:15207:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f1880 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449613615872:192.168.202.113@tcp:503:ll_cfg_requeue.0 00000100:00100000:0.0:1788571922.468711:0:15207:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada46424480 going to sleep for 11 seconds 00000100:00100000:0.0:1788571922.477117:0:15207:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f1880 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449613615872:192.168.202.113@tcp:503:ll_cfg_requeue.0 00000040:00080000:0.0:1788571922.478736:0:81349:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x2:0x0] index 64768 count 7 00010000:00010000:0.0:1788571924.343251:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788571925.367251:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788571927.415539:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788571928.439073:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:2.0:1788571928.595635:0:81400:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 0, flags 0, foreign 0 00000080:00200000:2.0:1788571928.595649:0:81400:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571928.595682:0:81400:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571928.595708:0:81400:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571928.595728:0:81400:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571928.595738:0:81400:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571928.595748:0:81400:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571928.595752:0:81400:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d651 00000080:00010000:2.0:1788571928.595763:0:81400:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d651 from it ffffab0ec1ec7cb8 00010000:00010000:2.0:1788571928.595768:0:81400:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571928.595779:0:81400:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571928.597224:0:81400:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571928.597253:0:81400:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571928.597274:0:81400:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571928.597291:0:81400:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571928.597300:0:81400:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571928.597307:0:81400:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571928.597312:0:81400:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d651 00000080:00010000:2.0:1788571928.597319:0:81400:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d651 from it ffffab0ec1ec7bb0 00010000:00010000:2.0:1788571928.597323:0:81400:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571928.597365:0:81400:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1788571928.597383:0:81400:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:2.0:1788571928.597396:0:81400:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200080:2.0:1788571928.597530:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:2.0:1788571928.597552:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a2ac00 uarg=0000000000000000 00800000:00000004:2.0:1788571928.597561:0:81400:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: age 3801, max_age 3845 00000100:00100000:2.0:1788571928.597625:0:81400:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37480 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:81400:1875449613617024:192.168.202.113@tcp:41:lfs.0 00000100:00100000:2.0:1788571928.597740:0:81400:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada61618a80 going to sleep for 11 seconds 00000100:00100000:3.0:1788571928.608287:0:81400:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37480 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:81400:1875449613617024:192.168.202.113@tcp:41:lfs.0 00800000:00000004:3.0:1788571928.608330:0:81400:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: update ffff9ada4481d198 cache blocks 317356/353529 objects 1023690/1024000 00000080:00200080:3.0:1788571928.608444:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.608469:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00800000:00000004:3.0:1788571928.608479:0:81400:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: age 3801, max_age 3845 00000100:00100000:3.0:1788571928.608527:0:81400:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a4ad80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:81400:1875449613617152:192.168.202.113@tcp:41:lfs.0 00000100:00100000:3.0:1788571928.608753:0:81400:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada5118a600 going to sleep for 11 seconds 00000100:00100000:3.0:1788571928.615658:0:81400:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4ad80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:81400:1875449613617152:192.168.202.113@tcp:41:lfs.0 00800000:00000004:3.0:1788571928.615703:0:81400:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: update ffff9ada44819e18 cache blocks 317400/353529 objects 1023737/1024000 00000080:00200080:3.0:1788571928.615815:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.615840:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.615850:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.615906:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.615913:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.615917:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.615920:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.615924:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.615928:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.615931:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.615936:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.615961:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.615969:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.615973:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.615977:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.615982:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.615986:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.615990:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.615994:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.617664:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.617687:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.617692:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.617696:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.617700:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.617703:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.617706:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.617710:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.617713:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.617717:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.617720:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.617724:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.617727:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.617731:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.617735:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.617738:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.617741:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.617745:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.617748:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.617752:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.617755:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.617758:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.617761:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.617765:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.617767:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.617770:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.617774:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.617777:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.617780:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.617783:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.617787:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.617790:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.617793:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.617797:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.618206:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.618215:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.618220:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.618225:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.618229:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.618234:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.618238:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.618241:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.618244:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.618248:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.618251:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.618255:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.618258:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.618262:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.618265:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.618269:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.618273:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.618277:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.618280:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.618284:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.618287:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.618290:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.618293:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.618297:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.618300:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.618304:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.618308:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.618312:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.618315:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.618318:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.618323:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.618327:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.618330:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.618334:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.618337:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.618341:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.618345:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.618349:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.618352:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.618357:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.618360:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.618364:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.618368:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.618372:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.618375:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.618379:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.618383:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.618387:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.618391:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.618394:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.618398:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.618402:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.618405:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.618409:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.618412:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.618416:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.618420:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.618424:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.618428:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.618431:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.618434:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.618438:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.618442:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.618446:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.618449:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.618453:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.618456:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.618460:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.618464:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.618468:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.618471:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.618474:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00800000:00000080:3.0:1788571928.618477:0:81400:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.618481:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.618488:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00020000:00000004:3.0:1788571928.618499:0:81400:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: age 3801, max_age 3845 00000100:00100000:3.0:1788571928.618583:0:81400:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a4ad80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:81400:1875449613617280:192.168.202.113@tcp:13:lfs.0 00000100:00100000:3.0:1788571928.618948:0:81400:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada5118a600 going to sleep for 11 seconds 00000100:00100000:3.0:1788571928.623276:0:81400:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4ad80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:81400:1875449613617280:192.168.202.113@tcp:13:lfs.0 00020000:00000004:3.0:1788571928.623318:0:81400:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: update ffff9ada4354d198 cache blocks 901357/958279 objects 261729/262144 00000080:00200080:3.0:1788571928.623412:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.623440:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00020000:00000004:3.0:1788571928.623448:0:81400:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: age 3801, max_age 3845 00000100:00100000:3.0:1788571928.623498:0:81400:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a4ad80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:81400:1875449613617408:192.168.202.113@tcp:13:lfs.0 00000100:00100000:3.0:1788571928.623788:0:81400:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada5118a600 going to sleep for 11 seconds 00000100:00100000:3.0:1788571928.627840:0:81400:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4ad80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:81400:1875449613617408:192.168.202.113@tcp:13:lfs.0 00020000:00000004:3.0:1788571928.627929:0:81400:0:(obd_class.h:1116:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: update ffff9ada43549e18 cache blocks 901363/958279 objects 261734/262144 00000080:00200080:3.0:1788571928.628035:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628059:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628068:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628073:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628077:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628080:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628084:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628088:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628091:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628095:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628100:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628103:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628107:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628111:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628115:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628119:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628123:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628128:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628132:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628136:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628140:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628144:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628148:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628152:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628156:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628159:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628163:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628166:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628170:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628174:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628178:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628181:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628185:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628188:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628192:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628196:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628211:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628215:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628219:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628222:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628226:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628229:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628234:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628238:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628242:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628246:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628250:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628253:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628257:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628260:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628264:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628267:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628270:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628273:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628277:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628281:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628285:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628288:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628292:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628296:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628300:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628302:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628306:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628309:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628313:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628316:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628320:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628325:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628329:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628333:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628337:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628340:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628344:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628348:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628352:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628356:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628360:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628364:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628367:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628370:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628373:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628376:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628380:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628384:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628387:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628391:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628395:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628398:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628402:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628406:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628410:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628414:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628418:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628422:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628426:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628430:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628434:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628438:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628442:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628446:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628450:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628454:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628458:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628461:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628465:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628468:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628472:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628476:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628479:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628483:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628487:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628491:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628494:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628498:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628513:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628518:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628523:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628527:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628530:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628533:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628537:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628540:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628544:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628547:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200080:3.0:1788571928.628550:0:81400:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd10ed65d0 00020000:00000080:3.0:1788571928.628553:0:81400:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada50b6a800 uarg=0000000000000000 00000080:00200008:3.0:1788571928.628571:0:81400:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00010000:00010000:0.0:1788571930.505205:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788571931.003670:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a34000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449613617536:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571931.003792:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a34700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613617792:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571931.003824:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613618048:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571931.003879:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1788571931.003886:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613617664:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571931.003917:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613617920:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571931.012519:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a35f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613617664:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571931.012544:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613617920:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571931.012555:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a34000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449613617536:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571931.012568:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a34700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613617792:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571931.012580:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a35180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613618048:192.168.202.113@tcp:400:kworker.0 00000080:00200008:0.0:1788571931.049799:0:81452:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1788571931.049820:0:81452:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571931.050294:0:81452:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571931.050321:0:81452:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1788571931.050341:0:81452:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1788571931.050351:0:81452:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788571931.050362:0:81452:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571931.050366:0:81452:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d651 00000080:00010000:0.0:1788571931.050377:0:81452:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d651 from it ffffab0ec0d1fcb8 00010000:00010000:0.0:1788571931.050382:0:81452:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1788571931.050392:0:81452:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200008:0.0:1788571931.051638:0:81452:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1788571931.051650:0:81452:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571931.051674:0:81452:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571931.051693:0:81452:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:0.0:1788571931.051707:0:81452:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:0.0:1788571931.051716:0:81452:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788571931.051722:0:81452:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571931.051726:0:81452:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d651 00000080:00010000:0.0:1788571931.051733:0:81452:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d651 from it ffffab0ec0d1fcb8 00010000:00010000:0.0:1788571931.051738:0:81452:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:0.0:1788571931.051746:0:81452:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00000080:00200000:0.0:1788571931.051930:0:81452:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571931.051939:0:81452:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571931.051957:0:81452:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:0.0:1788571931.051967:0:81452:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00000080:00200000:0.0:1788571931.051973:0:81452:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788571931.051978:0:81452:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571931.051980:0:81452:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d651 00000080:00010000:0.0:1788571931.051985:0:81452:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d651 from it ffffab0ec0d1fbb0 00010000:00010000:0.0:1788571931.052123:0:81452:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:0.0:1788571931.052131:0:81452:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00000080:00200000:0.0:1788571931.052142:0:81452:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:0.0:1788571931.052153:0:81452:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200080:0.0:1788571931.052323:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.052345:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00800000:00000004:0.0:1788571931.052355:0:81452:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: age 3846, max_age 3847 00000100:00100000:0.0:1788571931.052420:0:81452:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f2d80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:81452:1875449613618176:192.168.202.113@tcp:41:lfs.0 00000100:00100000:0.0:1788571931.052534:0:81452:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada48a7ab40 going to sleep for 11 seconds 00000100:00100000:0.0:1788571931.055963:0:81452:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f2d80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:81452:1875449613618176:192.168.202.113@tcp:41:lfs.0 00800000:00000004:0.0:1788571931.056012:0:81452:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: update ffff9ada4481d198 cache blocks 317356/353529 objects 1023690/1024000 00000080:00200080:0.0:1788571931.056106:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.056129:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00800000:00000004:0.0:1788571931.056140:0:81452:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: age 3846, max_age 3847 00000100:00100000:0.0:1788571931.056195:0:81452:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f2d80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:81452:1875449613618304:192.168.202.113@tcp:41:lfs.0 00000100:00100000:0.0:1788571931.056283:0:81452:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada48a7ab40 going to sleep for 11 seconds 00000100:00100000:0.0:1788571931.061676:0:81452:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f2d80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:81452:1875449613618304:192.168.202.113@tcp:41:lfs.0 00800000:00000004:0.0:1788571931.061714:0:81452:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: update ffff9ada44819e18 cache blocks 317400/353529 objects 1023737/1024000 00000080:00200080:0.0:1788571931.061835:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.061926:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.061937:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.061952:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.061958:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.061963:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.061970:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.061974:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.061978:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.061981:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062056:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062065:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062072:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062076:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062081:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062084:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062088:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062091:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062095:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062098:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062102:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062105:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062109:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062111:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062115:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062117:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062121:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062123:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062127:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062130:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062133:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062137:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062140:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062144:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062147:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062150:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062153:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062156:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062159:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062162:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062165:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062168:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062181:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062184:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062188:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062279:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062286:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062289:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062292:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062295:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062298:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062301:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062305:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062307:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062311:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062314:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062318:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062321:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062486:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062496:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062501:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062505:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062509:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062513:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062517:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062521:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062525:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062528:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062532:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062535:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062539:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062542:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062545:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062548:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062552:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062556:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062559:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062677:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062686:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062691:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062695:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062699:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062703:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062706:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062710:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062713:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062716:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062719:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062723:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062725:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062729:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062731:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062735:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062738:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062741:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062744:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062747:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062750:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062753:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062756:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062759:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062762:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062765:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062768:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062771:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062774:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062778:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062782:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062785:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062788:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062792:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062795:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062798:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062801:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062805:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062808:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062811:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062814:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062817:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062820:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062823:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062826:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062829:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062832:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062835:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00800000:00000080:0.0:1788571931.062838:0:81452:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.062842:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.062847:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00020000:00000004:0.0:1788571931.063310:0:81452:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: age 3846, max_age 3847 00000100:00100000:0.0:1788571931.063401:0:81452:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f2d80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:81452:1875449613618432:192.168.202.113@tcp:13:lfs.0 00000100:00100000:0.0:1788571931.063549:0:81452:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada48a7ab40 going to sleep for 11 seconds 00000100:00100000:0.0:1788571931.067707:0:81452:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f2d80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:81452:1875449613618432:192.168.202.113@tcp:13:lfs.0 00020000:00000004:0.0:1788571931.067748:0:81452:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: update ffff9ada4354d198 cache blocks 901357/958279 objects 261729/262144 00000080:00200080:0.0:1788571931.067845:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.067891:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00020000:00000004:0.0:1788571931.067902:0:81452:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: age 3846, max_age 3847 00000100:00100000:0.0:1788571931.067958:0:81452:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f2d80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:81452:1875449613618560:192.168.202.113@tcp:13:lfs.0 00000100:00100000:0.0:1788571931.068920:0:81452:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada48a7ab40 going to sleep for 11 seconds 00000100:00100000:0.0:1788571931.074162:0:81452:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f2d80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:81452:1875449613618560:192.168.202.113@tcp:13:lfs.0 00020000:00000004:0.0:1788571931.074329:0:81452:0:(obd_class.h:1116:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: update ffff9ada43549e18 cache blocks 901363/958279 objects 261734/262144 00000080:00200080:0.0:1788571931.074589:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074615:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074624:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074628:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074632:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074636:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074640:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074643:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074646:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074649:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074653:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074656:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074660:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074664:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074667:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074671:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074675:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074678:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074682:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074685:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074688:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074691:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074695:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074698:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074701:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074704:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074708:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074711:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074714:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074718:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074721:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074724:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074727:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074731:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074734:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074738:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074741:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074745:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074749:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074752:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074756:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074759:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074763:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074766:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074769:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074772:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074776:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074779:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074782:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074786:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074789:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074792:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074796:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074800:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074803:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074806:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074810:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074813:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074818:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074821:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074825:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074828:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074832:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074835:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074839:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074842:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074845:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074849:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074884:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074891:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074895:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074900:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074904:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074907:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074911:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074914:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074918:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074921:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074924:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074927:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074931:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074934:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074938:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074951:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074957:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074961:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074964:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074967:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074970:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074973:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074977:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.074980:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.074984:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.075159:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.075166:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.075179:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.075185:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.075188:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.075202:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.075206:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.075211:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.075215:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.075219:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.075222:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.075226:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.075229:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.075233:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.075236:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.075240:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.075244:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.075248:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.075251:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.075256:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.075261:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.075266:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.075270:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.075273:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.075277:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.075281:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.075284:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.075288:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.075291:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.075295:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.075299:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200080:0.0:1788571931.075302:0:81452:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe9a347cb0 00020000:00000080:0.0:1788571931.075306:0:81452:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a0000 uarg=0000000000000000 00000080:00200008:0.0:1788571931.075325:0:81452:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00010000:00010000:0.0:1788571931.526468:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788571933.560029:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788571934.583546:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788571936.632611:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788571937.655100:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:3.0:1788571940.478015:0:81590:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:3.0:1788571940.478063:0:81590:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1788571940.478092:0:81590:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00010000:00010000:3.0:1788571940.478116:0:81590:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00000080:00200000:3.0:1788571940.478126:0:81590:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1788571940.478136:0:81590:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:3.0:1788571940.478141:0:81590:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d651 00000080:00010000:3.0:1788571940.478152:0:81590:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d651 from it ffffab0ec1cafb58 00010000:00010000:3.0:1788571940.478158:0:81590:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00010000:00010000:3.0:1788571940.478175:0:81590:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00000080:00200000:3.0:1788571940.478195:0:81590:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:3.0:1788571940.478214:0:81590:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f35.replay-single, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), flags=8 00000080:00200000:3.0:1788571940.478220:0:81590:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f35.replay-single, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=getattr 00000002:00010000:3.0:1788571940.478238:0:81590:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f35.replay-single,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 01 00010000:00010000:3.0:1788571940.478301:0:81590:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763000/0xef30c340bfb3d658 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 81590 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788571940.478311:0:81590:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763000/0xef30c340bfb3d658 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 81590 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788571940.478323:0:81590:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763000/0xef30c340bfb3d658 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 81590 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788571940.478359:0:81590:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a4ad80 pname:cluuid:pid:xid:nid:opc:job checkstat:35eb2d60-2de0-4b00-9815-d5153edd0057:81590:1875449613619328:192.168.202.113@tcp:101:checkstat.0 00000100:00100000:3.0:1788571940.478484:0:81590:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada4522e6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788571940.491987:0:81590:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4ad80 pname:cluuid:pid:xid:nid:opc:job checkstat:35eb2d60-2de0-4b00-9815-d5153edd0057:81590:1875449613619328:192.168.202.113@tcp:101:checkstat.0 00010000:00010000:3.0:1788571940.492025:0:81590:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763000/0xef30c340bfb3d658 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 81590 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788571940.492046:0:81590:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763000/0xef30c340bfb3d658 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 81590 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788571940.492057:0:81590:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763000/0xef30c340bfb3d658 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 81590 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788571940.492071:0:81590:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763000/0xef30c340bfb3d658 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 81590 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788571940.492084:0:81590:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763000/0xef30c340bfb3d658 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 81590 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788571940.492091:0:81590:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada51763000 already unused, calling callback (ffffffffc1966630) 00010000:00010000:3.0:1788571940.492098:0:81590:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763000/0xef30c340bfb3d658 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 81590 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788571940.492107:0:81590:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763000/0xef30c340bfb3d658 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 81590 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788571940.492117:0:81590:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763000/0xef30c340bfb3d658 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 81590 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788571940.492125:0:81590:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1788571940.492133:0:81590:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763000/0xef30c340bfb3d658 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 81590 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788571940.492142:0:81590:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763000/0xef30c340bfb3d658 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 81590 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1788571940.492169:0:81590:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9ada60a4ad80 x1875449613619328/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/584 e 0 to 0 dl 1788571956 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'checkstat.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1788571940.492189:0:81590:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:3.0:1788571940.492232:0:81590:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00010000:00010000:3.0:1788571940.492249:0:81590:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00000080:00010000:3.0:1788571940.492273:0:81590:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d651 from it ffffab0ec1caf930 00010000:00010000:3.0:1788571940.492278:0:81590:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00010000:00010000:3.0:1788571940.492286:0:81590:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:3.0:1788571940.733979:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788571941.245264:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449613619456:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571941.245408:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613619712:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571941.259252:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613619712:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571941.259281:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449613619456:192.168.202.113@tcp:400:kworker.0 00010000:00010000:3.0:1788571941.751756:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571942.780424:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788571943.800080:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571945.846985:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788571946.364831:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613620096:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571946.364877:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613620352:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571946.364903:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613620224:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571946.364916:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613620480:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571946.365678:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449613619968:192.168.202.113@tcp:400:kworker.0 00010000:00010000:2.0:1788571946.872768:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788571947.904799:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571948.919665:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:0.0:1788571949.636369:0:82145:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: SKIP: replay-single test_36 skipping ALWAYS excluded test 36 00010000:00010000:2.0:1788571949.944056:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788571951.479097:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613620736:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571951.486454:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613620992:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571951.492610:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613620736:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571951.496212:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613620992:192.168.202.113@tcp:400:kworker.0 00000001:02000400:2.0:1788571951.887177:0:82296:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == replay-single test 37: abort recovery before client does replay (test mds_cleanup_orphans for directories) ========================================================== 21:32:30 (1788571950) 00010000:00010000:2.0:1788571951.992140:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:2.0:1788571951.994552:0:82298:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571951.994606:0:82298:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571951.994633:0:82298:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:2.0:1788571951.994653:0:82298:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00000080:00200000:2.0:1788571951.994663:0:82298:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571951.994671:0:82298:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571951.994675:0:82298:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d651 00000080:00010000:2.0:1788571951.994685:0:82298:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d651 from it ffffab0ec146bb58 00010000:00010000:2.0:1788571951.994689:0:82298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:2.0:1788571951.994798:0:82298:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00000080:00200000:2.0:1788571951.994818:0:82298:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:2.0:1788571951.994837:0:82298:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=d37.replay-single, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), flags=13 00000080:00200000:2.0:1788571951.994843:0:82298:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=d37.replay-single, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=getattr 00000002:00010000:2.0:1788571951.994904:0:82298:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: d37.replay-single,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 01 00010000:00010000:2.0:1788571951.994965:0:82298:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d65f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 82298 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571951.994975:0:82298:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d65f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 82298 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571951.994987:0:82298:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d65f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 82298 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571951.995020:0:82298:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35180 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:82298:1875449613621248:192.168.202.113@tcp:101:lfs.0 00000100:00100000:2.0:1788571951.995466:0:82298:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada47df3d80 going to sleep for 11 seconds 00000100:00100000:2.0:1788571951.999329:0:82298:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a35180 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:82298:1875449613621248:192.168.202.113@tcp:101:lfs.0 00010000:00010000:2.0:1788571951.999361:0:82298:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d65f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 82298 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571951.999379:0:82298:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d65f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 82298 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571951.999387:0:82298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d65f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 82298 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571951.999396:0:82298:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d65f lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 82298 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571951.999407:0:82298:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d65f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 82298 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571951.999413:0:82298:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada516e1600 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571951.999419:0:82298:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d65f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 82298 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571951.999426:0:82298:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d65f lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 82298 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571951.999434:0:82298:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d65f lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 82298 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571951.999442:0:82298:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571951.999450:0:82298:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d65f lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 82298 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571951.999458:0:82298:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb3d65f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 82298 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1788571951.999477:0:82298:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9ada60a35180 x1875449613621248/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/584 e 0 to 0 dl 1788571967 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1788571951.999495:0:82298:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:2.0:1788571951.999532:0:82298:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00010000:00010000:2.0:1788571951.999547:0:82298:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00000080:00010000:2.0:1788571951.999559:0:82298:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d651 from it ffffab0ec146b930 00010000:00010000:2.0:1788571951.999639:0:82298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00010000:00010000:2.0:1788571951.999650:0:82298:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00000080:00200000:2.0:1788571951.999805:0:82298:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571951.999823:0:82298:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571951.999831:0:82298:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00010000:00010000:2.0:1788571951.999847:0:82298:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00000080:00200000:2.0:1788571951.999900:0:82298:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571951.999906:0:82298:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571951.999910:0:82298:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d651 00000080:00010000:2.0:1788571951.999916:0:82298:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d651 from it ffffab0ec146bbe8 00010000:00010000:2.0:1788571951.999920:0:82298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00010000:00010000:2.0:1788571951.999928:0:82298:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00000080:00200000:2.0:1788571951.999937:0:82298:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:2.0:1788571951.999946:0:82298:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d37.replay-single, flags=16392 00000080:00200008:2.0:1788571951.999976:0:82298:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 0, flags 0, foreign 0 00000080:00200000:2.0:1788571951.999982:0:82298:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571951.999989:0:82298:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571951.999996:0:82298:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00010000:00010000:2.0:1788571952.000005:0:82298:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00000080:00200000:2.0:1788571952.000011:0:82298:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571952.000015:0:82298:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571952.000018:0:82298:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d651 00000080:00010000:2.0:1788571952.000022:0:82298:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d651 from it ffffab0ec146bcb8 00010000:00010000:2.0:1788571952.000025:0:82298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00010000:00010000:2.0:1788571952.000032:0:82298:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 12 lru_type: 1 00000080:00200008:2.0:1788571952.003277:0:82298:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 0, flags 0, foreign 0 00000080:00200000:2.0:1788571952.003291:0:82298:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571952.003321:0:82298:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571952.003345:0:82298:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 12 lru_type: 1 00010000:00010000:2.0:1788571952.003365:0:82298:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 12 lru_type: 1 00000080:00200000:2.0:1788571952.003375:0:82298:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571952.003384:0:82298:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571952.003389:0:82298:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d651 00000080:00010000:2.0:1788571952.003398:0:82298:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d651 from it ffffab0ec146bcb8 00010000:00010000:2.0:1788571952.003403:0:82298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 12 lru_type: 1 00010000:00010000:2.0:1788571952.003414:0:82298:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 13 lru_type: 1 00000080:00200000:2.0:1788571952.003663:0:82298:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571952.003674:0:82298:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571952.003683:0:82298:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 13 lru_type: 1 00010000:00010000:2.0:1788571952.003692:0:82298:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 13 lru_type: 1 00000080:00200000:2.0:1788571952.003767:0:82298:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571952.003773:0:82298:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571952.003776:0:82298:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d651 00000080:00010000:2.0:1788571952.003781:0:82298:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d651 from it ffffab0ec146bbb0 00010000:00010000:2.0:1788571952.003784:0:82298:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 13 lru_type: 1 00010000:00010000:2.0:1788571952.003792:0:82298:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d651 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7d69 expref: -99 pid: 81400 timeout: 0 lvb_type: 0 lru_score: 14 lru_type: 1 00000080:00200000:2.0:1788571952.003801:0:82298:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:2.0:1788571952.003810:0:82298:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 300000 00000080:00200080:2.0:1788571952.003833:0:82298:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c03066f0 arg=7ffc6c05eaa0 00000080:00200000:2.0:1788571952.003850:0:82298:0:(dir.c:451:ll_dir_setdirstripe()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) name=d37.replay-single stripe_offset=0 stripe_count=1, hash_type=0 00000100:00100000:2.0:1788571952.003972:0:82298:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35180 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:82298:1875449613621376:192.168.202.113@tcp:700:lfs.0 00000100:00100000:2.0:1788571952.004092:0:82298:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada47df3d80 going to sleep for 11 seconds 00000100:00100000:0.0:1788571952.050441:0:82298:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f7100 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:82298:1875449613621632:192.168.202.113@tcp:36:lfs.0 00000080:00200000:0.0:1788571952.050518:0:82298:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200017b01:0x1:0x0](ffff9ada58217448) 00000080:00200000:0.0:1788571952.050529:0:82298:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200017b01:0x1:0x0], updating i_size 4096 00000080:00200008:0.0:1788571952.050582:0:82298:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 300000 00000080:00200000:0.0:1788571952.070785:0:82304:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571952.070832:0:82304:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571952.071082:0:82304:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 82304 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571952.071096:0:82304:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 82304 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571952.071107:0:82304:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 82304 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571952.071140:0:82304:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0a80 pname:cluuid:pid:xid:nid:opc:job rmdir:35eb2d60-2de0-4b00-9815-d5153edd0057:82304:1875449613621760:192.168.202.113@tcp:101:rmdir.0 00000100:00100000:0.0:1788571952.072885:0:82304:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6073be40 going to sleep for 11 seconds 00000100:00100000:3.0:1788571952.082061:0:82304:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0a80 pname:cluuid:pid:xid:nid:opc:job rmdir:35eb2d60-2de0-4b00-9815-d5153edd0057:82304:1875449613621760:192.168.202.113@tcp:101:rmdir.0 00010000:00010000:3.0:1788571952.082097:0:82304:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788571952.082114:0:82304:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788571952.082128:0:82304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788571952.082139:0:82304:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788571952.082153:0:82304:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788571952.082163:0:82304:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788571952.082171:0:82304:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1788571952.082183:0:82304:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada516f0a80 x1875449613621760/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788571968 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1788571952.082200:0:82304:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788571952.082217:0:82304:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:3.0:1788571952.082226:0:82304:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1788571952.082259:0:82304:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:3.0:1788571952.082336:0:82304:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d666 00000080:00010000:3.0:1788571952.082343:0:82304:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:3.0:1788571952.082348:0:82304:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d666 from it ffffab0ec1cc7c20 00010000:00010000:3.0:1788571952.082351:0:82304:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788571952.082609:0:82304:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1788571952.082641:0:82304:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:3.0:1788571952.082667:0:82304:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=d37.replay-single, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), flags=24 00000080:00200000:3.0:1788571952.082673:0:82304:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=d37.replay-single, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=0 00000002:00010000:3.0:1788571952.082702:0:82304:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: d37.replay-single,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: lookup flags 0 00010000:00010000:3.0:1788571952.082758:0:82304:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763c00/0xef30c340bfb3d66d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 82304 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788571952.082767:0:82304:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763c00/0xef30c340bfb3d66d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 82304 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788571952.082778:0:82304:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763c00/0xef30c340bfb3d66d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 82304 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788571952.082804:0:82304:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a4ad80 pname:cluuid:pid:xid:nid:opc:job rmdir:35eb2d60-2de0-4b00-9815-d5153edd0057:82304:1875449613621888:192.168.202.113@tcp:101:rmdir.0 00000100:00100000:2.0:1788571952.086022:0:82304:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada5051ff00 going to sleep for 11 seconds 00000100:00100000:0.0:1788571952.103760:0:82304:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4ad80 pname:cluuid:pid:xid:nid:opc:job rmdir:35eb2d60-2de0-4b00-9815-d5153edd0057:82304:1875449613621888:192.168.202.113@tcp:101:rmdir.0 00010000:00010000:0.0:1788571952.103809:0:82304:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763c00/0xef30c340bfb3d66d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7dd2 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571952.103836:0:82304:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763c00/0xef30c340bfb3d66d lrc: 4/1,0 mode: --/PR res: [0x200017b01:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7dd2 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571952.103845:0:82304:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763c00/0xef30c340bfb3d66d lrc: 4/1,0 mode: --/PR res: [0x200017b01:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7dd2 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571952.103896:0:82304:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763c00/0xef30c340bfb3d66d lrc: 4/1,0 mode: PR/PR res: [0x200017b01:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7dd2 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571952.103909:0:82304:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763c00/0xef30c340bfb3d66d lrc: 4/1,0 mode: PR/PR res: [0x200017b01:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7dd2 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571952.103925:0:82304:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763c00/0xef30c340bfb3d66d lrc: 5/2,0 mode: PR/PR res: [0x200017b01:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7dd2 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571952.103933:0:82304:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763c00/0xef30c340bfb3d66d lrc: 5/2,0 mode: PR/PR res: [0x200017b01:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7dd2 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571952.103942:0:82304:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763c00/0xef30c340bfb3d66d lrc: 4/1,0 mode: PR/PR res: [0x200017b01:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7dd2 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788571952.103956:0:82304:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x10 disposition=0xb, status=0 req@ffff9ada60a4ad80 x1875449613621888/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/688 e 0 to 0 dl 1788571968 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788571952.103972:0:82304:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763c00/0xef30c340bfb3d66d lrc: 3/1,0 mode: PR/PR res: [0x200017b01:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7dd2 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571952.103986:0:82304:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8590031617/1 (0 0) 00000080:00200000:0.0:1788571952.103995:0:82304:0:(namei.c:952:ll_intent_lock()) intent lock 16 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1788571952.104034:0:82304:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200017b01:0x1:0x0], updating i_size 4096 00000080:00200000:0.0:1788571952.104038:0:82304:0:(namei.c:147:ll_iget()) got inode: [0x200017b01:0x1:0x0](ffff9ada58217448): rc = 0 00000080:00010000:0.0:1788571952.104043:0:82304:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200017b01:0x1:0x0](ffff9ada58217448) for lock 0xef30c340bfb3d66d 00000080:00010000:0.0:1788571952.104060:0:82304:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200017b01:0x1:0x0](ffff9ada58217448) 00000080:00010000:0.0:1788571952.104064:0:82304:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d66d from it ffffab0ec1cc7af0 00010000:00010000:0.0:1788571952.104067:0:82304:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763c00/0xef30c340bfb3d66d lrc: 3/1,0 mode: PR/PR res: [0x200017b01:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7dd2 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571952.104077:0:82304:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763c00/0xef30c340bfb3d66d lrc: 2/0,0 mode: PR/PR res: [0x200017b01:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7dd2 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571952.104273:0:82304:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200017b01:0x1:0x0](ffff9ada58217448), inode mode 41ed mask 1 00000080:00200000:0.0:1788571952.104300:0:82304:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f37.replay-single, dir=[0x200017b01:0x1:0x0](ffff9ada58217448), flags=0 00000080:00200000:0.0:1788571952.104305:0:82304:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f37.replay-single, dir=[0x200017b01:0x1:0x0](ffff9ada58217448), intent=getattr 00000002:00010000:0.0:1788571952.104332:0:82304:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f37.replay-single,[0x0:0x0:0x0]) in obj [0x200017b01:0x1:0x0], intent: getattr flags 01 00010000:00010000:0.0:1788571952.104385:0:82304:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d674 lrc: 3/1,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 82304 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571952.104394:0:82304:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d674 lrc: 3/1,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 82304 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571952.104404:0:82304:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d674 lrc: 3/1,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 82304 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571952.104434:0:82304:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f3800 pname:cluuid:pid:xid:nid:opc:job rmdir:35eb2d60-2de0-4b00-9815-d5153edd0057:82304:1875449613622016:192.168.202.113@tcp:101:rmdir.0 00000100:00100000:0.0:1788571952.104943:0:82304:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6073b540 going to sleep for 11 seconds 00000100:00100000:0.0:1788571952.120582:0:82304:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f3800 pname:cluuid:pid:xid:nid:opc:job rmdir:35eb2d60-2de0-4b00-9815-d5153edd0057:82304:1875449613622016:192.168.202.113@tcp:101:rmdir.0 00010000:00010000:0.0:1788571952.120618:0:82304:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d674 lrc: 4/1,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 82304 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571952.120637:0:82304:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d674 lrc: 4/1,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 82304 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571952.120657:0:82304:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d674 lrc: 4/1,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 82304 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571952.120665:0:82304:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d674 lrc: 3/0,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 82304 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571952.120677:0:82304:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d674 lrc: 4/0,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 82304 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571952.120684:0:82304:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada51617600 already unused, calling callback (ffffffffc1966630) 00010000:00010000:0.0:1788571952.120689:0:82304:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d674 lrc: 4/0,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 82304 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571952.120697:0:82304:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d674 lrc: 4/0,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 82304 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571952.120705:0:82304:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d674 lrc: 5/0,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 82304 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571952.120714:0:82304:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1788571952.120722:0:82304:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d674 lrc: 3/0,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 82304 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571952.120734:0:82304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d674 lrc: 0/0,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 82304 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788571952.120757:0:82304:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9ada516f3800 x1875449613622016/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/584 e 0 to 0 dl 1788571968 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'rmdir.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1788571952.120775:0:82304:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200017b01:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1788571952.120807:0:82304:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 30 res 8590031617/1 (0 0) 00000080:00200000:2.0:1788571952.179220:0:82310:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571952.179267:0:82310:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571952.179292:0:82310:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571952.179318:0:82310:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788571952.179328:0:82310:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571952.179336:0:82310:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571952.179341:0:82310:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d666 00000080:00010000:2.0:1788571952.179350:0:82310:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d666 from it ffffab0ec0ddbbe0 00010000:00010000:2.0:1788571952.179354:0:82310:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788571952.179364:0:82310:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571952.179384:0:82310:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:2.0:1788571952.179393:0:82310:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d37.replay-single, flags=24 00000080:00200000:2.0:1788571952.179412:0:82310:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200017b01:0x1:0x0](ffff9ada58217448), inode mode 41ed mask 1 00000080:00200000:2.0:1788571952.179427:0:82310:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f37.replay-single, dir=[0x200017b01:0x1:0x0](ffff9ada58217448), flags=1536 00000080:00200000:2.0:1788571952.179431:0:82310:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200017b01:0x1:0x0](ffff9ada58217448), inode mode 41ed mask 3 00000080:00200000:2.0:1788571952.179436:0:82310:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200017b01:0x1:0x0](ffff9ada58217448), inode mode 41ed mask 3 00000080:00200000:2.0:1788571952.179442:0:82310:0:(namei.c:2172:do_mkdir()) VFS Op:name=f37.replay-single, dir=[0x200017b01:0x1:0x0](ffff9ada58217448) 00000020:00100000:2.0:1788571952.179525:0:82310:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1788571952.179542:0:82310:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada60a34700 x1875449613622272/t0(0) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 568/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788571952.179572:0:82310:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a34700 pname:cluuid:pid:xid:nid:opc:job multiop:35eb2d60-2de0-4b00-9815-d5153edd0057:82310:1875449613622272:192.168.202.113@tcp:36:multiop.0 00000100:00100000:2.0:1788571952.179690:0:82310:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada47df3a80 going to sleep for 11 seconds 00000100:00100000:2.0:1788571952.190001:0:82310:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a34700 pname:cluuid:pid:xid:nid:opc:job multiop:35eb2d60-2de0-4b00-9815-d5153edd0057:82310:1875449613622272:192.168.202.113@tcp:36:multiop.0 00000080:00200000:2.0:1788571952.190086:0:82310:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200017b01:0x2:0x0](ffff9ada58191988) 00000080:00200000:2.0:1788571952.190096:0:82310:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200017b01:0x2:0x0], updating i_size 4096 00000080:00200000:2.0:1788571952.190166:0:82310:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571952.190189:0:82310:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571952.190214:0:82310:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571952.190230:0:82310:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788571952.190240:0:82310:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571952.190247:0:82310:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571952.190251:0:82310:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d666 00000080:00010000:2.0:1788571952.190259:0:82310:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d666 from it ffffab0ec0ddbbf8 00010000:00010000:2.0:1788571952.190264:0:82310:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788571952.190274:0:82310:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1788571952.190293:0:82310:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:2.0:1788571952.190300:0:82310:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d37.replay-single, flags=27 00000080:00200000:2.0:1788571952.190306:0:82310:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200017b01:0x1:0x0](ffff9ada58217448), inode mode 41ed mask 1 00000080:00200000:2.0:1788571952.190319:0:82310:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f37.replay-single, dir=[0x200017b01:0x1:0x0](ffff9ada58217448), file ffff9ada50cd8d80, open_flags 18000, mode 0 opened 0 00000080:00200000:2.0:1788571952.190327:0:82310:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f37.replay-single, dir=[0x200017b01:0x1:0x0](ffff9ada58217448), intent=open 00000002:00010000:2.0:1788571952.190345:0:82310:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f37.replay-single,[0x0:0x0:0x0]) in obj [0x200017b01:0x1:0x0], intent: open flags 0300001 00010000:00010000:2.0:1788571952.190415:0:82310:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d67b lrc: 3/1,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 82310 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571952.190426:0:82310:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d67b lrc: 3/1,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 82310 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1788571952.190438:0:82310:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1788571952.190449:0:82310:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada60a34000 x1875449613622528/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/131856 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1788571952.190460:0:82310:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d67b lrc: 3/1,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 82310 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571952.190482:0:82310:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a34000 pname:cluuid:pid:xid:nid:opc:job multiop:35eb2d60-2de0-4b00-9815-d5153edd0057:82310:1875449613622528:192.168.202.113@tcp:101:multiop.0 00000100:00100000:2.0:1788571952.190725:0:82310:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada47df3a80 going to sleep for 11 seconds 00000100:00100000:2.0:1788571952.194370:0:82310:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a34000 pname:cluuid:pid:xid:nid:opc:job multiop:35eb2d60-2de0-4b00-9815-d5153edd0057:82310:1875449613622528:192.168.202.113@tcp:101:multiop.0 00010000:00010000:2.0:1788571952.194403:0:82310:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d67b lrc: 4/1,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 82310 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571952.194420:0:82310:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d67b lrc: 4/1,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 82310 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571952.194429:0:82310:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d67b lrc: 4/1,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 82310 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571952.194436:0:82310:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d67b lrc: 3/0,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 82310 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571952.194447:0:82310:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d67b lrc: 4/0,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 82310 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571952.194457:0:82310:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada516e1000 already unused, calling callback (ffffffffc1966630) 00010000:00010000:2.0:1788571952.194463:0:82310:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d67b lrc: 4/0,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 82310 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571952.194469:0:82310:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d67b lrc: 4/0,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 82310 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571952.194477:0:82310:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d67b lrc: 5/0,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x20e400000000 pid: 82310 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571952.194484:0:82310:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788571952.194491:0:82310:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d67b lrc: 3/0,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 82310 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571952.194498:0:82310:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d67b lrc: 0/0,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 82310 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1788571952.194520:0:82310:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x1 disposition=0x2b, status=0 req@ffff9ada60a34000 x1875449613622528/t206158430211(206158430211) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/608 e 0 to 0 dl 1788571968 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'multiop.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1788571952.194533:0:82310:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada60a34000 x1875449613622528/t206158430211(206158430211) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/608 e 0 to 0 dl 1788571968 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'multiop.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1788571952.194548:0:82310:0:(namei.c:952:ll_intent_lock()) intent lock 1 on i1 [0x200017b01:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1788571952.194577:0:82310:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200017b01:0x2:0x0], updating i_size 4096 00000080:00200000:2.0:1788571952.194581:0:82310:0:(namei.c:147:ll_iget()) got inode: [0x200017b01:0x2:0x0](ffff9ada58191988): rc = 0 00000080:00010000:2.0:1788571952.194585:0:82310:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200017b01:0x2:0x0](ffff9ada58191988) for lock 0x0 00000080:00200008:2.0:1788571952.194616:0:82310:0:(file.c:1090:ll_file_open()) START file f37.replay-single:[0x200017b01:0x2:0x0](ffff9ada58191988), flags 300000 00000002:00100000:2.0:1788571952.194638:0:82310:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada60a34000 x1875449613622528/t206158430211(206158430211) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/608 e 0 to 0 dl 1788571968 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'multiop.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1788571952.194694:0:82310:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200017b01:0x2:0x0](ffff9ada58191988), inode mode 41ed mask 44 00000080:00200000:0.0:1788571952.230708:0:82313:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571952.230753:0:82313:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571952.230779:0:82313:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1788571952.230800:0:82313:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1788571952.230809:0:82313:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788571952.230820:0:82313:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571952.230824:0:82313:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d666 00000080:00010000:0.0:1788571952.230835:0:82313:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d666 from it ffffab0ec0dabc20 00010000:00010000:0.0:1788571952.230839:0:82313:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1788571952.230851:0:82313:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:0.0:1788571952.230909:0:82313:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:0.0:1788571952.230919:0:82313:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d37.replay-single, flags=24 00000080:00200000:0.0:1788571952.230927:0:82313:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200017b01:0x1:0x0](ffff9ada58217448), inode mode 41ed mask 1 00000080:00200000:0.0:1788571952.230943:0:82313:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f37.replay-single, dir=[0x200017b01:0x1:0x0](ffff9ada58217448), flags=0 00000080:00200000:0.0:1788571952.230949:0:82313:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f37.replay-single, dir=[0x200017b01:0x1:0x0](ffff9ada58217448), intent=getattr 00000002:00010000:0.0:1788571952.230965:0:82313:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f37.replay-single,[0x0:0x0:0x0]) in obj [0x200017b01:0x1:0x0], intent: getattr flags 01 00010000:00010000:0.0:1788571952.231025:0:82313:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d682 lrc: 3/1,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 82313 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571952.231034:0:82313:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d682 lrc: 3/1,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 82313 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571952.231046:0:82313:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d682 lrc: 3/1,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 82313 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571952.231081:0:82313:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f3800 pname:cluuid:pid:xid:nid:opc:job rmdir:35eb2d60-2de0-4b00-9815-d5153edd0057:82313:1875449613622656:192.168.202.113@tcp:101:rmdir.0 00000100:00100000:0.0:1788571952.231208:0:82313:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6073b600 going to sleep for 11 seconds 00000100:00100000:0.0:1788571952.240263:0:82313:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f3800 pname:cluuid:pid:xid:nid:opc:job rmdir:35eb2d60-2de0-4b00-9815-d5153edd0057:82313:1875449613622656:192.168.202.113@tcp:101:rmdir.0 00010000:00010000:0.0:1788571952.240299:0:82313:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d682 lrc: 4/1,0 mode: --/CR res: [0x200017b01:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7e34 expref: -99 pid: 82313 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571952.240334:0:82313:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d682 lrc: 4/1,0 mode: --/PR res: [0x200017b01:0x2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7e34 expref: -99 pid: 82313 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571952.240344:0:82313:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d682 lrc: 4/1,0 mode: --/PR res: [0x200017b01:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7e34 expref: -99 pid: 82313 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571952.240372:0:82313:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d682 lrc: 4/1,0 mode: PR/PR res: [0x200017b01:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7e34 expref: -99 pid: 82313 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571952.240385:0:82313:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d682 lrc: 4/1,0 mode: PR/PR res: [0x200017b01:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7e34 expref: -99 pid: 82313 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571952.240403:0:82313:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d682 lrc: 5/2,0 mode: PR/PR res: [0x200017b01:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7e34 expref: -99 pid: 82313 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571952.240410:0:82313:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d682 lrc: 5/2,0 mode: PR/PR res: [0x200017b01:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7e34 expref: -99 pid: 82313 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571952.240418:0:82313:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d682 lrc: 4/1,0 mode: PR/PR res: [0x200017b01:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7e34 expref: -99 pid: 82313 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788571952.240431:0:82313:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada516f3800 x1875449613622656/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/584 e 0 to 0 dl 1788571968 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rmdir.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788571952.240446:0:82313:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d682 lrc: 3/1,0 mode: PR/PR res: [0x200017b01:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7e34 expref: -99 pid: 82313 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571952.240458:0:82313:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8590031617/2 (0 0) 00000080:00200000:0.0:1788571952.240466:0:82313:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200017b01:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1788571952.240494:0:82313:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200017b01:0x2:0x0], updating i_size 4096 00000080:00200000:0.0:1788571952.240498:0:82313:0:(namei.c:147:ll_iget()) got inode: [0x200017b01:0x2:0x0](ffff9ada58191988): rc = 0 00000080:00010000:0.0:1788571952.240505:0:82313:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200017b01:0x2:0x0](ffff9ada58191988) for lock 0xef30c340bfb3d682 00000080:00010000:0.0:1788571952.240526:0:82313:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200017b01:0x2:0x0](ffff9ada58191988) 00000080:00010000:0.0:1788571952.240531:0:82313:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d682 from it ffffab0ec0dabe08 00010000:00010000:0.0:1788571952.240534:0:82313:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d682 lrc: 3/1,0 mode: PR/PR res: [0x200017b01:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7e34 expref: -99 pid: 82313 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571952.240544:0:82313:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d682 lrc: 2/0,0 mode: PR/PR res: [0x200017b01:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7e34 expref: -99 pid: 82313 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571952.240599:0:82313:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200017b01:0x1:0x0](ffff9ada58217448), inode mode 41ed mask 3 00000080:00200000:0.0:1788571952.240611:0:82313:0:(namei.c:2261:ll_rmdir()) VFS Op:name=f37.replay-single, dir=[0x200017b01:0x1:0x0](ffff9ada58217448) 00010000:00010000:0.0:1788571952.240647:0:82313:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d682 lrc: 2/0,0 mode: PR/PR res: [0x200017b01:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b82f7e34 expref: -99 pid: 82313 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571952.240707:0:82313:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d682 lrc: 1/0,0 mode: --/PR res: [0x200017b01:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f7e34 expref: -99 pid: 82313 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571952.240715:0:82313:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1788571952.240719:0:82313:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51617600/0xef30c340bfb3d682 lrc: 0/0,0 mode: --/PR res: [0x200017b01:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f7e34 expref: -99 pid: 82313 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1788571952.240745:0:82313:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1788571952.240753:0:82313:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada516f3800 x1875449613622912/t0(0) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788571952.240781:0:82313:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f3800 pname:cluuid:pid:xid:nid:opc:job rmdir:35eb2d60-2de0-4b00-9815-d5153edd0057:82313:1875449613622912:192.168.202.113@tcp:36:rmdir.0 00000100:00100000:0.0:1788571952.241071:0:82313:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6073b600 going to sleep for 11 seconds 00000100:00100000:0.0:1788571952.249121:0:82313:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f3800 pname:cluuid:pid:xid:nid:opc:job rmdir:35eb2d60-2de0-4b00-9815-d5153edd0057:82313:1875449613622912:192.168.202.113@tcp:36:rmdir.0 00010000:00010000:2.0:1788571953.018287:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:2.0:1788571953.297538:0:82322:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1788571953.297560:0:82322:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571953.297591:0:82322:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571953.297617:0:82322:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:2.0:1788571953.297636:0:82322:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:2.0:1788571953.297644:0:82322:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571953.297653:0:82322:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571953.297657:0:82322:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d666 00000080:00010000:2.0:1788571953.297668:0:82322:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d666 from it ffffab0ec1c37cb8 00010000:00010000:2.0:1788571953.297673:0:82322:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:2.0:1788571953.297683:0:82322:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00000080:00200008:2.0:1788571953.299586:0:82322:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 0, flags 0, foreign 0 00000080:00200000:2.0:1788571953.299597:0:82322:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571953.299622:0:82322:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571953.299643:0:82322:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:2.0:1788571953.299664:0:82322:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00000080:00200000:2.0:1788571953.299673:0:82322:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571953.299682:0:82322:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571953.299685:0:82322:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d666 00000080:00010000:2.0:1788571953.299693:0:82322:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d666 from it ffffab0ec1c37cb8 00010000:00010000:2.0:1788571953.299697:0:82322:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:2.0:1788571953.299706:0:82322:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00000080:00200000:2.0:1788571953.299884:0:82322:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571953.299896:0:82322:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571953.299904:0:82322:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00010000:00010000:2.0:1788571953.299912:0:82322:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00000080:00200000:2.0:1788571953.299919:0:82322:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571953.299922:0:82322:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571953.299925:0:82322:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d666 00000080:00010000:2.0:1788571953.299929:0:82322:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d666 from it ffffab0ec1c37bb0 00010000:00010000:2.0:1788571953.299932:0:82322:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00010000:00010000:2.0:1788571953.299940:0:82322:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 0 00000080:00200000:2.0:1788571953.299948:0:82322:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:2.0:1788571953.299958:0:82322:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200080:2.0:1788571953.300090:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:2.0:1788571953.300596:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28000 uarg=0000000000000000 00800000:00000004:2.0:1788571953.300607:0:82322:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: age 3848, max_age 3869 00000100:00100000:2.0:1788571953.300667:0:82322:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37480 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:82322:1875449613623040:192.168.202.113@tcp:41:lfs.0 00000100:00100000:2.0:1788571953.300783:0:82322:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada47df3a80 going to sleep for 11 seconds 00000080:00200080:0.0:1788571953.314781:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.314815:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00800000:00000004:0.0:1788571953.314826:0:82322:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: age 3848, max_age 3869 00000100:00100000:0.0:1788571953.314919:0:82322:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0e00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:82322:1875449613623168:192.168.202.113@tcp:41:lfs.0 00000100:00100000:0.0:1788571953.315080:0:82322:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6073b600 going to sleep for 11 seconds 00000100:00100000:0.0:1788571953.325920:0:82322:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0e00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:82322:1875449613623168:192.168.202.113@tcp:41:lfs.0 00800000:00000004:0.0:1788571953.325968:0:82322:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: update ffff9ada44819e18 cache blocks 317400/353529 objects 1023737/1024000 00000080:00200080:0.0:1788571953.326052:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326076:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326085:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326089:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326093:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326097:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326102:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326105:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326122:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326127:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326131:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326135:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326139:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326142:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326146:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326149:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326152:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326155:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326159:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326162:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326165:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326168:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326172:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326176:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326180:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326184:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326187:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326191:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326195:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326198:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326203:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326207:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326211:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326214:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326217:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326222:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326226:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326229:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326233:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326235:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326239:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326242:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326245:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326248:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326252:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326255:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326260:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326263:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326267:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326271:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326275:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326278:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326283:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326286:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326291:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326294:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326298:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326302:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326306:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326310:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326314:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326317:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326322:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326325:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326330:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326333:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326338:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326341:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326345:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326349:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326352:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326356:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326359:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326363:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326367:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326370:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326374:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326378:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326381:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326384:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326388:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326391:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326395:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326398:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326402:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326405:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326409:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326412:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326416:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326419:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326424:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326427:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326431:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326435:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326439:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326443:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326447:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326450:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326454:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326459:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326463:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326467:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326470:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326474:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326478:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326481:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326485:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326489:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326493:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326495:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326499:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326502:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326505:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326508:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326511:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326514:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326518:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326521:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326524:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326527:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326531:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326535:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326538:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326542:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326546:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00800000:00000080:0.0:1788571953.326550:0:82322:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.326554:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.326561:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00020000:00000004:0.0:1788571953.326571:0:82322:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: age 3848, max_age 3869 00000100:00100000:0.0:1788571953.326623:0:82322:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0e00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:82322:1875449613623296:192.168.202.113@tcp:13:lfs.0 00000100:00100000:0.0:1788571953.326731:0:82322:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6073b600 going to sleep for 11 seconds 00000100:00100000:0.0:1788571953.332103:0:82322:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0e00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:82322:1875449613623296:192.168.202.113@tcp:13:lfs.0 00020000:00000004:0.0:1788571953.332163:0:82322:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: update ffff9ada4354d198 cache blocks 901357/958279 objects 261729/262144 00000080:00200080:0.0:1788571953.332283:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.332310:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00020000:00000004:0.0:1788571953.332322:0:82322:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: age 3848, max_age 3870 00000100:00100000:0.0:1788571953.332369:0:82322:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0e00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:82322:1875449613623424:192.168.202.113@tcp:13:lfs.0 00000100:00100000:0.0:1788571953.332476:0:82322:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6073b600 going to sleep for 11 seconds 00000100:00100000:0.0:1788571953.335026:0:82322:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0e00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:82322:1875449613623424:192.168.202.113@tcp:13:lfs.0 00020000:00000004:0.0:1788571953.335066:0:82322:0:(obd_class.h:1116:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: update ffff9ada43549e18 cache blocks 901363/958279 objects 261734/262144 00000080:00200080:0.0:1788571953.335162:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335188:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335196:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335201:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335205:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335209:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335212:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335216:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335221:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335225:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335229:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335232:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335236:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335239:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335243:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335246:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335250:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335253:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335257:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335261:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335265:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335268:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335273:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335276:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335280:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335284:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335288:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335291:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335295:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335298:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335302:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335306:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335310:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335314:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335318:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335321:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335325:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335329:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335332:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335336:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335339:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335342:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335345:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335349:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335353:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335356:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335359:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335363:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335366:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335369:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335373:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335376:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335379:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335382:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335386:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335389:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335393:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335396:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335401:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335404:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335408:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335411:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335416:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335419:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335423:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335427:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335430:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335433:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335437:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335441:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335445:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335448:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335453:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335456:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335462:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335465:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335469:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335473:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335476:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335493:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335499:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335503:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335508:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335512:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335517:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335520:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335524:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335528:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335533:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335537:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335541:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335544:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335548:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335552:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335557:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335561:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335565:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335569:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335573:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335577:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335581:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335584:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335588:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335591:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335594:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335598:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335601:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335605:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335609:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335612:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335616:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335620:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335624:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335627:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335632:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335635:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335640:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335643:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335646:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335650:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335654:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335657:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335661:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335664:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200080:0.0:1788571953.335668:0:82322:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe462073b0 00020000:00000080:0.0:1788571953.335671:0:82322:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2800 uarg=0000000000000000 00000080:00200008:0.0:1788571953.335691:0:82322:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200000:0.0:1788571953.461727:0:82329:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571953.461769:0:82329:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571953.461796:0:82329:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 0 00010000:00010000:0.0:1788571953.462306:0:82329:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 0 00000080:00200000:0.0:1788571953.462325:0:82329:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788571953.462338:0:82329:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571953.462342:0:82329:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d666 00000080:00010000:0.0:1788571953.462353:0:82329:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d666 from it ffffab0ec1d17bd0 00010000:00010000:0.0:1788571953.462358:0:82329:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 0 00010000:00010000:0.0:1788571953.462369:0:82329:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00000080:00200000:0.0:1788571953.462393:0:82329:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:0.0:1788571953.462419:0:82329:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=fsa-oleg213-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), flags=1536 00000080:00200000:0.0:1788571953.462423:0:82329:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571953.462433:0:82329:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571953.462442:0:82329:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:0.0:1788571953.462451:0:82329:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00000080:00200000:0.0:1788571953.462457:0:82329:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788571953.462461:0:82329:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571953.462464:0:82329:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d666 00000080:00010000:0.0:1788571953.462468:0:82329:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d666 from it ffffab0ec1d17cd0 00010000:00010000:0.0:1788571953.462471:0:82329:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:0.0:1788571953.462751:0:82329:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00000080:00200000:0.0:1788571953.462762:0:82329:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:0.0:1788571953.462771:0:82329:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571953.462781:0:82329:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571953.462788:0:82329:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00010000:00010000:0.0:1788571953.462808:0:82329:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00000080:00200000:0.0:1788571953.462815:0:82329:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788571953.462820:0:82329:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571953.462822:0:82329:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d666 00000080:00010000:0.0:1788571953.462827:0:82329:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d666 from it ffffab0ec1d17db0 00010000:00010000:0.0:1788571953.462830:0:82329:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00010000:00010000:0.0:1788571953.462838:0:82329:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00000080:00200000:0.0:1788571953.462847:0:82329:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:0.0:1788571953.462901:0:82329:0:(namei.c:2062:ll_create_nd()) VFS Op:name=fsa-oleg213-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), flags=33188, excl=1 00000080:00200000:0.0:1788571953.462907:0:82329:0:(namei.c:2017:ll_mknod()) VFS Op:name=fsa-oleg213-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9ada58125b88) mode 100644 dev 0 00010000:00010000:0.0:1788571953.462943:0:82329:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00010000:00010000:0.0:1788571953.463013:0:82329:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00010000:00010000:0.0:1788571953.463021:0:82329:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1788571953.463024:0:82329:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d666 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f7db6 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00000020:00100000:0.0:1788571953.463058:0:82329:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1788571953.463075:0:82329:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada516f0e00 x1875449613623680/t0(0) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788571953.463104:0:82329:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0e00 pname:cluuid:pid:xid:nid:opc:job mcreate:35eb2d60-2de0-4b00-9815-d5153edd0057:82329:1875449613623680:192.168.202.113@tcp:36:mcreate.0 00000100:00100000:0.0:1788571953.463352:0:82329:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6073b600 going to sleep for 11 seconds 00000100:00100000:0.0:1788571953.468359:0:82329:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0e00 pname:cluuid:pid:xid:nid:opc:job mcreate:35eb2d60-2de0-4b00-9815-d5153edd0057:82329:1875449613623680:192.168.202.113@tcp:36:mcreate.0 00000080:00200000:0.0:1788571953.468432:0:82329:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x200017b01:0x3:0x0](ffff9ada58214b08) 00000080:00200000:0.0:1788571953.468491:0:82329:0:(namei.c:2070:ll_create_nd()) VFS Op:name=fsa-oleg213-client.virtnet, unhashed 1 00000080:00200000:2.0:1788571953.493294:0:82332:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571953.493337:0:82332:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571953.493424:0:82332:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d689 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 82332 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571953.493438:0:82332:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d689 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 82332 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571953.493451:0:82332:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d689 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 82332 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571953.493494:0:82332:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35f80 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:82332:1875449613623808:192.168.202.113@tcp:101:rm.0 00000100:00100000:2.0:1788571953.493614:0:82332:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada47df3900 going to sleep for 11 seconds 00000100:00100000:0.0:1788571953.504453:0:82332:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a35f80 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:82332:1875449613623808:192.168.202.113@tcp:101:rm.0 00010000:00010000:0.0:1788571953.504503:0:82332:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d689 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7e65 expref: -99 pid: 82332 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571953.504519:0:82332:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d689 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7e65 expref: -99 pid: 82332 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571953.504532:0:82332:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d689 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7e65 expref: -99 pid: 82332 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571953.504549:0:82332:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d689 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7e65 expref: -99 pid: 82332 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571953.504565:0:82332:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d689 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7e65 expref: -99 pid: 82332 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571953.504573:0:82332:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d689 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7e65 expref: -99 pid: 82332 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571953.504580:0:82332:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d689 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7e65 expref: -99 pid: 82332 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788571953.504593:0:82332:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada60a35f80 x1875449613623808/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788571969 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788571953.504608:0:82332:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d689 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7e65 expref: -99 pid: 82332 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571953.504622:0:82332:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1788571953.504630:0:82332:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1788571953.504667:0:82332:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 20480 00000080:00010000:0.0:1788571953.504674:0:82332:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d689 00000080:00010000:0.0:1788571953.504681:0:82332:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571953.504685:0:82332:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d689 from it ffffab0ec0dabb58 00010000:00010000:0.0:1788571953.504689:0:82332:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d689 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7e65 expref: -99 pid: 82332 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571953.504705:0:82332:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d689 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7e65 expref: -99 pid: 82332 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571953.504741:0:82332:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:0.0:1788571953.504771:0:82332:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=fsa-oleg213-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), flags=8 00000080:00200000:0.0:1788571953.504777:0:82332:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=fsa-oleg213-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=getattr 00000002:00010000:0.0:1788571953.504813:0:82332:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: fsa-oleg213-client.virtnet,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 01 00010000:00010000:0.0:1788571953.504921:0:82332:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d690 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 82332 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571953.504929:0:82332:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d690 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 82332 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571953.504940:0:82332:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d690 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 82332 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788571953.504966:0:82332:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0700 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:82332:1875449613623936:192.168.202.113@tcp:101:rm.0 00000100:00100000:0.0:1788571953.505066:0:82332:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6073b600 going to sleep for 11 seconds 00000100:00100000:0.0:1788571953.511552:0:82332:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0700 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:82332:1875449613623936:192.168.202.113@tcp:101:rm.0 00010000:00010000:0.0:1788571953.511593:0:82332:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d690 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7e81 expref: -99 pid: 82332 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571953.511620:0:82332:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d690 lrc: 4/1,0 mode: --/PR res: [0x200017b01:0x3:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7e81 expref: -99 pid: 82332 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571953.511630:0:82332:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d690 lrc: 4/1,0 mode: --/PR res: [0x200017b01:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7e81 expref: -99 pid: 82332 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571953.511645:0:82332:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d690 lrc: 4/1,0 mode: PR/PR res: [0x200017b01:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7e81 expref: -99 pid: 82332 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571953.511657:0:82332:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d690 lrc: 4/1,0 mode: PR/PR res: [0x200017b01:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7e81 expref: -99 pid: 82332 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571953.511672:0:82332:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d690 lrc: 5/2,0 mode: PR/PR res: [0x200017b01:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7e81 expref: -99 pid: 82332 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571953.511681:0:82332:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d690 lrc: 5/2,0 mode: PR/PR res: [0x200017b01:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7e81 expref: -99 pid: 82332 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571953.511690:0:82332:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d690 lrc: 4/1,0 mode: PR/PR res: [0x200017b01:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7e81 expref: -99 pid: 82332 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788571953.511705:0:82332:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada516f0700 x1875449613623936/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 600/584 e 0 to 0 dl 1788571969 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788571953.511719:0:82332:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d690 lrc: 3/1,0 mode: PR/PR res: [0x200017b01:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7e81 expref: -99 pid: 82332 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571953.511732:0:82332:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8590031617/3 (0 0) 00000080:00200000:0.0:1788571953.511741:0:82332:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1788571953.511771:0:82332:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200017b01:0x3:0x0], updating i_size 0 00000080:00200000:0.0:1788571953.511775:0:82332:0:(namei.c:147:ll_iget()) got inode: [0x200017b01:0x3:0x0](ffff9ada58214b08): rc = 0 00000080:00200000:0.0:1788571953.511796:0:82332:0:(file.c:6843:ll_layout_conf()) [0x200017b01:0x3:0x0]: layout version change: 4294967294 -> 4294967295 00000080:00010000:0.0:1788571953.511812:0:82332:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200017b01:0x3:0x0](ffff9ada58214b08) for lock 0xef30c340bfb3d690 00000080:00010000:0.0:1788571953.511829:0:82332:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200017b01:0x3:0x0](ffff9ada58214b08) 00000080:00010000:0.0:1788571953.511834:0:82332:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d690 from it ffffab0ec0dabb20 00010000:00010000:0.0:1788571953.511838:0:82332:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d690 lrc: 3/1,0 mode: PR/PR res: [0x200017b01:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f7e81 expref: -99 pid: 82332 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788571953.511851:0:82332:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d690 lrc: 2/0,0 mode: PR/PR res: [0x200017b01:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f7e81 expref: -99 pid: 82332 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:0.0:1788571953.511955:0:82332:0:(file.c:6296:ll_getattr_dentry()) START file fsa-oleg213-client.virtnet:[0x200017b01:0x3:0x0](ffff9ada58214b08), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1788571953.511963:0:82332:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200017b01:0x3:0x0](ffff9ada58214b08),name=fsa-oleg213-client.virtnet 00000002:00010000:0.0:1788571953.511994:0:82332:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200017b01:0x3:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571953.512005:0:82332:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d690 lrc: 2/1,0 mode: PR/PR res: [0x200017b01:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f7e81 expref: -99 pid: 82332 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571953.512018:0:82332:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d690 lrc: 2/1,0 mode: PR/PR res: [0x200017b01:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f7e81 expref: -99 pid: 82332 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571953.512026:0:82332:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788571953.512031:0:82332:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200017b01:0x3:0x0](ffff9ada58214b08) 00000080:00010000:0.0:1788571953.512034:0:82332:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200017b01:0x3:0x0](ffff9ada58214b08) for lock 0xef30c340bfb3d690 00000080:00010000:0.0:1788571953.512038:0:82332:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d690 from it ffffab0ec0dabcb8 00010000:00010000:0.0:1788571953.512041:0:82332:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d690 lrc: 3/1,0 mode: PR/PR res: [0x200017b01:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b82f7e81 expref: -99 pid: 82332 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571953.512048:0:82332:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d690 lrc: 2/0,0 mode: PR/PR res: [0x200017b01:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b82f7e81 expref: -99 pid: 82332 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788571953.512346:0:82332:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571953.512357:0:82332:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571953.512365:0:82332:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d689 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7e65 expref: -99 pid: 82332 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571953.512375:0:82332:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d689 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7e65 expref: -99 pid: 82332 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788571953.512381:0:82332:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788571953.512385:0:82332:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571953.512388:0:82332:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d689 00000080:00010000:0.0:1788571953.512392:0:82332:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d689 from it ffffab0ec0dabc10 00010000:00010000:0.0:1788571953.512395:0:82332:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d689 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7e65 expref: -99 pid: 82332 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571953.512403:0:82332:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d689 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7e65 expref: -99 pid: 82332 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788571953.512414:0:82332:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:0.0:1788571953.512425:0:82332:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=fsa-oleg213-client.virtnet, flags=0 00000080:00200000:0.0:1788571953.512431:0:82332:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788571953.512438:0:82332:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788571953.512444:0:82332:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d689 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7e65 expref: -99 pid: 82332 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571953.512452:0:82332:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d689 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7e65 expref: -99 pid: 82332 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788571953.512459:0:82332:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788571953.512463:0:82332:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788571953.512466:0:82332:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d689 00000080:00010000:0.0:1788571953.512469:0:82332:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d689 from it ffffab0ec0dabd68 00010000:00010000:0.0:1788571953.512472:0:82332:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d689 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f7e65 expref: -99 pid: 82332 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571953.512489:0:82332:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d689 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f7e65 expref: -99 pid: 82332 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1788571953.512497:0:82332:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:0.0:1788571953.512504:0:82332:0:(namei.c:2351:ll_unlink()) VFS Op:name=fsa-oleg213-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9ada58125b88) 00010000:00010000:0.0:1788571953.512530:0:82332:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d689 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b82f7e65 expref: -99 pid: 82332 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1788571953.512558:0:82332:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d690 lrc: 2/0,0 mode: PR/PR res: [0x200017b01:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0xdb7edb52b82f7e81 expref: -99 pid: 82332 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788571953.512569:0:82332:0:(vvp_object.c:117:vvp_conf_set()) [0x200017b01:0x3:0x0]: losing layout lock 00010000:00010000:0.0:1788571953.512613:0:82332:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d690 lrc: 1/0,0 mode: --/PR res: [0x200017b01:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0xdb7edb52b82f7e81 expref: -99 pid: 82332 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571953.512621:0:82332:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d689 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f7e65 expref: -99 pid: 82332 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1788571953.512627:0:82332:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:0.0:1788571953.512630:0:82332:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51616600/0xef30c340bfb3d690 lrc: 0/0,0 mode: --/PR res: [0x200017b01:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0xdb7edb52b82f7e81 expref: -99 pid: 82332 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788571953.512655:0:82332:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb3d689 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b82f7e65 expref: -99 pid: 82332 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:0.0:1788571953.512684:0:82332:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1788571953.512695:0:82332:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada516f0700 x1875449613624192/t0(0) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 512/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788571953.512726:0:82332:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0700 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:82332:1875449613624192:192.168.202.113@tcp:36:rm.0 00000100:00100000:0.0:1788571953.513054:0:82332:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6073b600 going to sleep for 11 seconds 00000100:00100000:0.0:1788571953.536292:0:82332:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0700 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:82332:1875449613624192:192.168.202.113@tcp:36:rm.0 00000080:00200000:0.0:1788571953.536380:0:82332:0:(vvp_io.c:1826:vvp_io_init()) [0x200017b01:0x3:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:0.0:1788571953.536394:0:82332:0:(vvp_io.c:304:vvp_io_fini()) [0x200017b01:0x3:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1788571953.536404:0:82332:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200017b01:0x3:0x0](ffff9ada58214b08) 00010000:00010000:2.0:1788571955.063076:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571956.087080:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788571956.599304:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449613624320:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571956.604394:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613624576:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571956.604515:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613624832:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571956.620430:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613624576:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571956.620455:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613624832:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571956.624460:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449613624320:192.168.202.113@tcp:400:kworker.0 00010000:00010000:2.0:1788571958.135056:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571959.159943:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571961.207019:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:0.0:1788571961.574267:0:82399:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: local REPLAY BARRIER on lustre-MDT0000 00010000:00010000:2.0:1788571962.232341:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571964.279047:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571965.303142:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788571966.841338:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613625728:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571966.841476:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613625984:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571966.841517:0:2383:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:0.0:1788571966.841525:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449613625600:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571966.841557:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613625856:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571966.841583:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613626112:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571966.845019:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613625856:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571966.845044:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613626112:192.168.202.113@tcp:400:kworker.0 00000100:00080000:0.0:1788571966.845057:0:2383:0:(recover.c:205:ptlrpc_request_handle_notconn()) import lustre-MDT0000-mdc-ffff9ada58b0a000 of lustre-MDT0000_UUID@192.168.202.113@tcp abruptly disconnected: reconnecting 00000100:00080000:0.0:1788571966.845064:0:2383:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:0.0:1788571966.845069:0:2383:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff9ada58b0a000: Connection to lustre-MDT0000 (at 192.168.202.113@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:0.0:1788571966.845078:0:2383:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788571966.845084:0:2383:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 3838 00000100:00080000:0.0:1788571966.845097:0:2383:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788571966.845400:0:2383:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788571966.845442:0:2383:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada516f2300 x1875449613626240/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788571966.845524:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449613626240:192.168.202.113@tcp:38:ptlrpcd_00_00.0 00000100:00100000:0.0:1788571966.845629:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613625728:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571966.845646:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613625984:192.168.202.113@tcp:400:kworker.0 00000100:00080000:3.0:1788571966.852513:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788571966.852534:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-19) 00000100:00100000:3.0:1788571966.852540:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449613626240:192.168.202.113@tcp:38:ptlrpcd_00_00.0 00010000:00010000:2.0:1788571967.352032:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571968.376571:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571970.423103:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571971.448735:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788571971.959187:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788571971.959197:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788571971.959202:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788571971.959216:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 3884 00000100:00080000:2.0:1788571971.959231:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 5s 00000100:00080000:2.0:1788571971.959240:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788571971.959254:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788571971.959277:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60a37100 x1875449613626496/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788571971.959367:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a36300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613626624:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571971.959500:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613626880:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571971.959538:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1788571971.959545:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a36680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449613626368:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788571971.959576:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613626752:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571971.959817:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449613626496:192.168.202.113@tcp:38:kworker.0 00000100:00080000:0.0:1788571971.971202:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788571971.971224:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-19) 00000100:00100000:0.0:1788571971.971231:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449613626496:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788571971.972335:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a36300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613626624:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788571971.977254:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a35500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613626752:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788571971.977278:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449613626880:192.168.202.113@tcp:400:kworker.0 00010000:00010000:2.0:1788571973.498069:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571974.519116:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571976.567250:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788571977.079191:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788571977.079200:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788571977.079206:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788571977.079218:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 3889 00000100:00080000:0.0:1788571977.079232:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 10s 00000100:00080000:0.0:1788571977.079242:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788571977.079256:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788571977.079278:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada516f1500 x1875449613627136/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788571977.079372:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613627264:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788571977.079392:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449613627136:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788571977.079497:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613627520:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571977.108916:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613627264:192.168.202.113@tcp:400:kworker.0 00010000:00010000:2.0:1788571977.591059:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788571978.617130:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571979.644336:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571980.669305:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788571981.687085:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788571982.199186:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788571982.199197:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788571982.199202:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788571982.199226:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 3894 00000100:00080000:0.0:1788571982.199242:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 15s 00000100:00080000:0.0:1788571982.199251:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788571982.199267:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788571982.199292:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada516f0a80 x1875449613627776/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788571982.199745:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449613627776:192.168.202.113@tcp:38:kworker.0 00000100:00080000:0.0:1788571982.202893:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788571982.202911:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788571982.202917:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788571982.202931:0:77466:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 3838 00000100:00080000:0.0:1788571982.202950:0:77466:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788571982.202994:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada516f2d80 x1875449613628288/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788571982.209447:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a36680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449613626368:192.168.202.113@tcp:400:kworker.0 00000100:00080000:3.0:1788571982.214723:0:2382:0:(import.c:1176:ptlrpc_connect_interpret()) MGC192.168.202.113@tcp: connect to target with instance 0 10000000:01000000:3.0:1788571982.214745:0:2382:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808005 00000100:02000400:3.0:1788571982.214756:0:2382:0:(import.c:1328:ptlrpc_connect_interpret()) Evicted from MGS (at 192.168.202.113@tcp) after server handle changed from 0xdb7edb52b82f7d1c to 0xdb7edb52b82f81a6 00000100:00080000:3.0:1788571982.214762:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to EVICTED 00000100:00080000:3.0:1788571982.214770:0:2382:0:(import.c:1718:ptlrpc_import_recovery_state_machine()) evicted from MGS@192.168.202.113@tcp; invalidating 00000100:00100000:0.0:1788571982.215455:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449613628288:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788571982.215630:0:82630:0:(import.c:1642:ptlrpc_invalidate_import_thread()) thread invalidate import MGC192.168.202.113@tcp to MGS@192.168.202.113@tcp 00000100:00100000:3.0:1788571982.215635:0:82630:0:(import.c:323:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 10000000:01000000:3.0:1788571982.215639:0:82630:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808003 00010000:00010000:3.0:1788571982.215659:0:82630:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.202.113@tcp lock: ffff9ada53016800/0xef30c340bfb3d63c lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x110e400000000 nid: local remote: 0xdb7edb52b82f7d2a expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1788571982.215683:0:82630:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.202.113@tcp lock: ffff9ada53016800/0xef30c340bfb3d63c lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x110f400000000 nid: local remote: 0xdb7edb52b82f7d2a expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1788571982.215690:0:82630:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:3.0:1788571982.215711:0:82630:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1788571982.215723:0:82630:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.202.113@tcp lock: ffff9ada53016800/0xef30c340bfb3d63c lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x590f400000000 nid: local remote: 0xdb7edb52b82f7d2a expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788571982.215739:0:82630:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.202.113@tcp lock: ffff9ada53017200/0xef30c340bfb3d64a lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0xdb7edb52b82f7d38 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1788571982.215745:0:82630:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.202.113@tcp lock: ffff9ada53017200/0xef30c340bfb3d64a lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0xdb7edb52b82f7d38 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1788571982.215750:0:82630:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:3.0:1788571982.215754:0:82630:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1788571982.215757:0:82630:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.202.113@tcp lock: ffff9ada53017200/0xef30c340bfb3d64a lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0xdb7edb52b82f7d38 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788571982.215770:0:82630:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.202.113@tcp lock: ffff9ada53016200/0xef30c340bfb3d635 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0xdb7edb52b82f7d23 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1788571982.215776:0:82630:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.202.113@tcp lock: ffff9ada53016200/0xef30c340bfb3d635 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0xdb7edb52b82f7d23 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1788571982.215785:0:82630:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:3.0:1788571982.215789:0:82630:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1788571982.215792:0:82630:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.202.113@tcp lock: ffff9ada53016200/0xef30c340bfb3d635 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0xdb7edb52b82f7d23 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788571982.215815:0:82630:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.202.113@tcp lock: ffff9ada51617a00/0xef30c340bfb3d643 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0xdb7edb52b82f7d31 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1788571982.215823:0:82630:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.202.113@tcp lock: ffff9ada51617a00/0xef30c340bfb3d643 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0xdb7edb52b82f7d31 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1788571982.215828:0:82630:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:3.0:1788571982.215831:0:82630:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1788571982.215835:0:82630:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.202.113@tcp lock: ffff9ada51617a00/0xef30c340bfb3d643 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0xdb7edb52b82f7d31 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1788571982.215903:0:82630:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from EVICTED to RECOVER 00000100:00080000:3.0:1788571982.215909:0:82630:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from RECOVER to FULL 10000000:01000000:3.0:1788571982.215911:0:82630:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808004 00000100:00080000:3.0:1788571982.215915:0:82630:0:(pinger.c:186:ptlrpc_pinger_ir_up()) IR up 00000100:02000000:3.0:1788571982.215919:0:82630:0:(import.c:1787:ptlrpc_import_recovery_state_machine()) MGC192.168.202.113@tcp: Connection restored to 192.168.202.113@tcp (at 192.168.202.113@tcp) 00010000:00010000:2.0:1788571982.218829:0:15205:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 0 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1788571982.219017:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571982.219020:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571982.219022:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00080000:0.0:1788571982.226135:0:2382:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff9ada516f0a80 x1875449613627776/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/416 e 0 to 0 dl 1788572007 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788571982.226169:0:2382:0:(import.c:1176:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to target with instance 49 00000100:00080000:0.0:1788571982.226286:0:2382:0:(import.c:1382:ptlrpc_connect_interpret()) @@@ lustre-MDT0000-mdc-ffff9ada58b0a000: evicting (reconnect/recover flags not set: 4) req@ffff9ada516f0a80 x1875449613627776/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/416 e 0 to 0 dl 1788572007 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788571982.226296:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to EVICTED 00000100:02020000:0.0:1788571982.226302:0:2382:0:(import.c:1707:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff9ada58b0a000: This client was evicted by lustre-MDT0000; in progress operations using this service will fail. 00000100:00080000:0.0:1788571982.274650:0:2382:0:(import.c:1718:ptlrpc_import_recovery_state_machine()) evicted from lustre-MDT0000_UUID@192.168.202.113@tcp; invalidating 00000100:00100000:0.0:1788571982.279741:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449613627776:192.168.202.113@tcp:38:kworker.0 00000100:00080000:0.0:1788571982.279799:0:82631:0:(import.c:1642:ptlrpc_invalidate_import_thread()) thread invalidate import lustre-MDT0000-mdc-ffff9ada58b0a000 to lustre-MDT0000_UUID@192.168.202.113@tcp 00000100:00080000:0.0:1788571982.279805:0:82631:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import lustre-MDT0000_UUID INVALID 00000100:00100000:0.0:1788571982.279811:0:82631:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 206158430212 gen 9 00000100:00100000:0.0:1788571982.279821:0:82631:0:(client.c:3164:ptlrpc_free_committed()) @@@ free request with old gen req@ffff9ada516f0e00 x1875449613623680/t206158430213(206158430213) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788571969 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788571982.279835:0:82631:0:(client.c:3164:ptlrpc_free_committed()) @@@ free request with old gen req@ffff9ada516f0700 x1875449613624192/t206158430214(206158430214) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 512/456 e 0 to 0 dl 1788571969 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788571982.279848:0:82631:0:(client.c:3214:ptlrpc_free_committed()) @@@ free stale open request req@ffff9ada60a34000 x1875449613622528/t206158430211(206158430211) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/608 e 0 to 0 dl 1788571968 ref 1 fl Complete:RPQU/604/ffffffff rc 301/-1 job:'multiop.0' uid:0 gid:0 projid:0 00000100:00100000:0.0:1788571982.279892:0:82631:0:(import.c:323:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 00010000:00010000:0.0:1788571982.279955:0:82631:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763c00/0xef30c340bfb3d66d lrc: 3/0,0 mode: PR/PR res: [0x200017b01:0x1:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x10e400000000 nid: local remote: 0xdb7edb52b82f7dd2 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788571982.279985:0:82631:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1788571982.280001:0:82631:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763c00/0xef30c340bfb3d66d lrc: 0/0,0 mode: --/PR res: [0x200017b01:0x1:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x490f400000000 nid: local remote: 0xdb7edb52b82f7dd2 expref: -99 pid: 82304 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000100:00080000:0.0:1788571982.281928:0:82631:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from EVICTED to RECOVER 00000100:00080000:0.0:1788571982.281941:0:82631:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from RECOVER to FULL 00000100:02000000:3.0:1788571982.294495:0:82631:0:(import.c:1787:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff9ada58b0a000: Connection restored to 192.168.202.113@tcp (at 192.168.202.113@tcp) 00010000:00010000:2.0:1788571982.711068:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788571983.735660:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:2.0:1788571983.991457:0:15207:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:2.0:1788571983.991469:0:15207:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9ada58b0a000 from 196 10000000:01000000:2.0:1788571983.991476:0:15207:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:2.0:1788571983.991548:0:15207:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.202.113@tcp lock: ffff9ada516e1600/0xef30c340bfb3d697 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571983.991572:0:15207:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.202.113@tcp lock: ffff9ada516e1600/0xef30c340bfb3d697 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571983.991585:0:15207:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.202.113@tcp lock: ffff9ada516e1600/0xef30c340bfb3d697 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571983.991620:0:15207:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35c00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449613628416:192.168.202.113@tcp:101:ll_cfg_requeue.0 00000100:00100000:2.0:1788571983.991743:0:15207:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada47df3a80 going to sleep for 11 seconds 00000020:01000000:2.0:1788571984.065490:0:15207:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 52-51 (rc=0) 10000000:01000000:2.0:1788571984.065537:0:15207:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.202.113@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:2.0:1788571984.065556:0:15207:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.202.113@tcp lock: ffff9ada53017200/0xef30c340bfb3d69e lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b82f8278 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571984.065614:0:15207:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.202.113@tcp lock: ffff9ada53017200/0xef30c340bfb3d69e lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b82f8278 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1788571984.065628:0:15207:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:2.0:1788571984.065633:0:15207:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9ada58b0a000 from 7 10000000:01000000:2.0:1788571984.065637:0:15207:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:2.0:1788571984.065700:0:15207:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.202.113@tcp lock: ffff9ada516e1c00/0xef30c340bfb3d6a5 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571984.065710:0:15207:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.202.113@tcp lock: ffff9ada516e1c00/0xef30c340bfb3d6a5 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571984.065723:0:15207:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.202.113@tcp lock: ffff9ada516e1c00/0xef30c340bfb3d6a5 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571984.065759:0:15207:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35880 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449613629056:192.168.202.113@tcp:101:ll_cfg_requeue.0 00000100:00100000:2.0:1788571984.067161:0:15207:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada47df3d80 going to sleep for 11 seconds 00000020:01000000:2.0:1788571984.091793:0:15207:0:(obd_config.c:2145:class_config_parse_llog()) Processed log params gen 7-6 (rc=0) 10000000:01000000:2.0:1788571984.091832:0:15207:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.202.113@tcp: configuration from log 'params' succeeded (0). 00010000:00010000:2.0:1788571984.091847:0:15207:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.202.113@tcp lock: ffff9ada516e1c00/0xef30c340bfb3d6a5 lrc: 3/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b82f827f expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571984.091893:0:15207:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.202.113@tcp lock: ffff9ada516e1c00/0xef30c340bfb3d6a5 lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b82f827f expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1788571984.091906:0:15207:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:2.0:1788571984.091909:0:15207:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9ada42e899c0 from 1 10000000:01000000:2.0:1788571984.091912:0:15207:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:2.0:1788571984.091972:0:15207:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.202.113@tcp lock: ffff9ada516e0800/0xef30c340bfb3d6ac lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571984.091980:0:15207:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.202.113@tcp lock: ffff9ada516e0800/0xef30c340bfb3d6ac lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571984.091992:0:15207:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.202.113@tcp lock: ffff9ada516e0800/0xef30c340bfb3d6ac lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788571984.092027:0:15207:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a34e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449613629440:192.168.202.113@tcp:101:ll_cfg_requeue.0 00000100:00100000:2.0:1788571984.092148:0:15207:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada47df3900 going to sleep for 11 seconds 00000100:00100000:3.0:1788571984.129250:0:15207:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f6680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449613629568:192.168.202.113@tcp:501:ll_cfg_requeue.0 10000000:01000000:3.0:1788571984.129332:0:15207:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.202.113@tcp: configuration from log 'lustre-sptlrpc' failed (-2). 00010000:00010000:3.0:1788571984.129346:0:15207:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.202.113@tcp lock: ffff9ada516e0800/0xef30c340bfb3d6ac lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b82f8286 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788571984.129363:0:15207:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.202.113@tcp lock: ffff9ada516e0800/0xef30c340bfb3d6ac lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b82f8286 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788571984.759135:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571985.783115:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788571986.808302:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571987.832256:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571988.855073:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788571989.880651:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571990.903100:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200080:0.0:1788571991.162694:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.162737:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.162755:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.162760:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.162764:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.162768:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.162773:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.162776:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.162780:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.162784:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.162788:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.162791:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.162795:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.162812:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.162818:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.162821:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.162826:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.162829:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.162834:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.162837:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.162841:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.162845:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.162849:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.162887:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.162894:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.162897:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.162901:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.162905:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.162909:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.162912:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.162916:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.162921:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.162925:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.162929:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.162933:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.162937:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.162940:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.162944:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.162947:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.162951:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.162955:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.162958:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.162962:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.162966:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.162969:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.162973:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.162978:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.162981:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.162985:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.162988:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.162991:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.162994:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.162997:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.163001:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.163005:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.163008:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.163012:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.163016:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.163020:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.163024:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.163027:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.163031:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.163035:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.163038:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.163042:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.163045:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.163049:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.163053:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.163069:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.163073:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.163077:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.163080:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.163085:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.163089:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.163093:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.163097:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.163102:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.163106:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.163111:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.163114:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.163119:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.163123:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.163128:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.163132:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.163136:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.163140:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.163143:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.163146:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.163149:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.163152:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.163155:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.163159:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.163163:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.163166:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.163170:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.163174:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.163178:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.163181:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.163184:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.163188:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.163192:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.163195:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.163199:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.163203:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.163207:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.163211:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.163215:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.163219:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.163223:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.163227:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.163232:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.163235:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.163239:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.163242:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.163246:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.163249:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.163253:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.163256:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.163260:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.163263:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.163266:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.163270:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.163274:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.163276:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.163280:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00800000:00000080:0.0:1788571991.163284:0:82721:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.163289:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.163296:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00020000:00000004:0.0:1788571991.163307:0:82721:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: age 3871, max_age 3907 00000100:00100000:0.0:1788571991.163741:0:82721:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f1880 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:82721:1875449613630720:192.168.202.113@tcp:13:lfs.0 00000100:00100000:0.0:1788571991.163913:0:82721:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6050d840 going to sleep for 11 seconds 00000100:00100000:0.0:1788571991.167581:0:82721:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f1880 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:82721:1875449613630720:192.168.202.113@tcp:13:lfs.0 00020000:00000004:0.0:1788571991.167638:0:82721:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: update ffff9ada4354d198 cache blocks 901357/958279 objects 261729/262144 00000080:00200080:0.0:1788571991.167747:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.167770:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00020000:00000004:0.0:1788571991.167782:0:82721:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: age 3871, max_age 3907 00000100:00100000:0.0:1788571991.167825:0:82721:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f1880 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:82721:1875449613630848:192.168.202.113@tcp:13:lfs.0 00000100:00100000:0.0:1788571991.167969:0:82721:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6050d840 going to sleep for 11 seconds 00000100:00100000:0.0:1788571991.172234:0:82721:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f1880 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:82721:1875449613630848:192.168.202.113@tcp:13:lfs.0 00020000:00000004:0.0:1788571991.172274:0:82721:0:(obd_class.h:1116:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: update ffff9ada43549e18 cache blocks 901363/958279 objects 261734/262144 00000080:00200080:0.0:1788571991.172361:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172384:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172392:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172398:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172402:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172406:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172410:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172413:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172417:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172420:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172424:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172427:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172431:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172445:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172451:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172456:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172460:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172464:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172467:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172470:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172474:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172478:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172481:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172485:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172488:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172492:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172496:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172500:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172504:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172507:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172511:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172515:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172519:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172522:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172527:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172530:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172534:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172537:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172542:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172546:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172551:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172554:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172559:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172563:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172567:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172571:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172575:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172579:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172583:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172587:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172591:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172595:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172599:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172603:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172607:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172610:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172614:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172617:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172632:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172637:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172641:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172645:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172648:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172652:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172656:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172660:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172664:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172668:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172672:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172676:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172679:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172683:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172687:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172691:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172695:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172698:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172702:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172705:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172709:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172712:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172716:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172720:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172723:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172726:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172730:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172733:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172737:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172740:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172744:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172747:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172751:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172754:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172758:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172762:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172766:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172770:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172774:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172778:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172782:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172785:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172789:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172792:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172796:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172812:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172817:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172820:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172824:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172828:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172832:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172835:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172840:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172843:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172847:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172851:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172906:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172913:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172917:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172920:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172924:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172928:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172932:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172936:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172939:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172942:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200080:0.0:1788571991.172946:0:82721:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd54632c80 00020000:00000080:0.0:1788571991.172950:0:82721:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a1c00 uarg=0000000000000000 00000080:00200008:0.0:1788571991.172968:0:82721:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00010000:00010000:2.0:1788571991.928227:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788571992.444448:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a4b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449613630976:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788571992.444578:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a4bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613631232:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788571992.444611:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a49880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613631488:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788571992.444662:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1788571992.444668:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a4a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613631104:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788571992.445314:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a4ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613631360:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788571992.448120:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613631360:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788571992.448140:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449613630976:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788571992.448153:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a49880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613631488:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788571992.449368:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613631104:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788571992.449444:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449613631232:192.168.202.113@tcp:400:kworker.0 00010000:00010000:0.0:1788571992.951765:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:2.0:1788571993.171288:0:82773:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1788571993.171307:0:82773:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571993.171337:0:82773:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571993.171364:0:82773:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d6b3 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f82b7 expref: -99 pid: 82721 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1788571993.171386:0:82773:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d6b3 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f82b7 expref: -99 pid: 82721 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1788571993.171397:0:82773:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571993.171417:0:82773:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571993.171421:0:82773:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d6b3 00000080:00010000:2.0:1788571993.171431:0:82773:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d6b3 from it ffffab0ec0d53cb8 00010000:00010000:2.0:1788571993.171436:0:82773:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d6b3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f82b7 expref: -99 pid: 82721 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:2.0:1788571993.171446:0:82773:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d6b3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f82b7 expref: -99 pid: 82721 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200008:2.0:1788571993.172485:0:82773:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 0, flags 0, foreign 0 00000080:00200000:2.0:1788571993.172493:0:82773:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571993.172508:0:82773:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571993.172522:0:82773:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d6b3 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f82b7 expref: -99 pid: 82721 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:2.0:1788571993.172534:0:82773:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d6b3 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f82b7 expref: -99 pid: 82721 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:2.0:1788571993.172542:0:82773:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571993.172548:0:82773:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571993.172551:0:82773:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d6b3 00000080:00010000:2.0:1788571993.172557:0:82773:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d6b3 from it ffffab0ec0d53cb8 00010000:00010000:2.0:1788571993.172561:0:82773:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d6b3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f82b7 expref: -99 pid: 82721 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:2.0:1788571993.172570:0:82773:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d6b3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f82b7 expref: -99 pid: 82721 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00000080:00200000:2.0:1788571993.173486:0:82773:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788571993.173507:0:82773:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788571993.173521:0:82773:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d6b3 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f82b7 expref: -99 pid: 82721 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:2.0:1788571993.173532:0:82773:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d6b3 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f82b7 expref: -99 pid: 82721 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00000080:00200000:2.0:1788571993.173544:0:82773:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788571993.173549:0:82773:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788571993.173551:0:82773:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb3d6b3 00000080:00010000:2.0:1788571993.173555:0:82773:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb3d6b3 from it ffffab0ec0d53bb0 00010000:00010000:2.0:1788571993.173558:0:82773:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d6b3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b82f82b7 expref: -99 pid: 82721 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:2.0:1788571993.173567:0:82773:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada53017600/0xef30c340bfb3d6b3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b82f82b7 expref: -99 pid: 82721 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00000080:00200000:2.0:1788571993.173578:0:82773:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:2.0:1788571993.173651:0:82773:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200080:2.0:1788571993.173785:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.173804:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00800000:00000004:2.0:1788571993.173813:0:82773:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: age 3908, max_age 3909 00000100:00100000:2.0:1788571993.174476:0:82773:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a36a00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:82773:1875449613631616:192.168.202.113@tcp:41:lfs.0 00000100:00100000:2.0:1788571993.174607:0:82773:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6073b600 going to sleep for 11 seconds 00000100:00100000:2.0:1788571993.177749:0:82773:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a36a00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:82773:1875449613631616:192.168.202.113@tcp:41:lfs.0 00800000:00000004:2.0:1788571993.177788:0:82773:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: update ffff9ada4481d198 cache blocks 317347/353529 objects 1023688/1024000 00000080:00200080:2.0:1788571993.178075:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.178097:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00800000:00000004:2.0:1788571993.178107:0:82773:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: age 3908, max_age 3909 00000100:00100000:2.0:1788571993.178151:0:82773:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a36a00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:82773:1875449613631744:192.168.202.113@tcp:41:lfs.0 00000100:00100000:2.0:1788571993.178282:0:82773:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6073b600 going to sleep for 11 seconds 00000100:00100000:2.0:1788571993.181040:0:82773:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a36a00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:82773:1875449613631744:192.168.202.113@tcp:41:lfs.0 00800000:00000004:2.0:1788571993.181079:0:82773:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: update ffff9ada44819e18 cache blocks 317392/353529 objects 1023736/1024000 00000080:00200080:2.0:1788571993.181223:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181250:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181259:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181264:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181269:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181273:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181277:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181281:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181284:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181288:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181292:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181295:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181300:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181303:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181308:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181311:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181316:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181320:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181325:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181328:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181333:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181336:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181341:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181346:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181349:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181353:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181357:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181360:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181364:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181367:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181372:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181375:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181379:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181383:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181387:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181391:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181395:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181398:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181414:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181418:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181423:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181426:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181430:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181434:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181438:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181442:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181445:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181449:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181455:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181459:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181463:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181467:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181471:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181475:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181479:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181483:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181487:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181491:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181494:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181498:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181501:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181505:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181509:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181513:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181517:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181520:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181524:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181528:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181532:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181535:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181540:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181543:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181548:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181553:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181557:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181561:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181565:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181569:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181574:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181578:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181581:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181585:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181662:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181668:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181673:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181676:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181681:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181684:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181688:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181691:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181695:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181698:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181702:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181706:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181710:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181713:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181717:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181721:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181726:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181729:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181734:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181737:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181741:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181745:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181748:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181751:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181755:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181758:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181761:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181764:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181768:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181771:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181775:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181777:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181782:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181785:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181788:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181791:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181795:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181798:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181802:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181806:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181810:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181813:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181817:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00800000:00000080:2.0:1788571993.181820:0:82773:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00000080:00200080:2.0:1788571993.181824:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:2.0:1788571993.181831:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada42a28400 uarg=0000000000000000 00020000:00000004:2.0:1788571993.181842:0:82773:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: age 3908, max_age 3909 00000100:00100000:2.0:1788571993.181935:0:82773:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a36a00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:82773:1875449613631872:192.168.202.113@tcp:13:lfs.0 00000100:00100000:2.0:1788571993.182035:0:82773:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6073b600 going to sleep for 11 seconds 00000100:00100000:0.0:1788571993.203910:0:82773:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a36a00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:82773:1875449613631872:192.168.202.113@tcp:13:lfs.0 00020000:00000004:0.0:1788571993.203953:0:82773:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: update ffff9ada4354d198 cache blocks 901357/958279 objects 261729/262144 00000080:00200080:0.0:1788571993.204049:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.204072:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00020000:00000004:0.0:1788571993.204081:0:82773:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: age 3908, max_age 3909 00000100:00100000:0.0:1788571993.204128:0:82773:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f1880 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:82773:1875449613632000:192.168.202.113@tcp:13:lfs.0 00000100:00100000:0.0:1788571993.204207:0:82773:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6050d6c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788571993.211958:0:82773:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f1880 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:82773:1875449613632000:192.168.202.113@tcp:13:lfs.0 00020000:00000004:0.0:1788571993.211997:0:82773:0:(obd_class.h:1116:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: update ffff9ada43549e18 cache blocks 901363/958279 objects 261734/262144 00000080:00200080:0.0:1788571993.212103:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212127:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212136:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212141:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212145:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212150:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212154:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212158:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212162:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212166:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212170:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212174:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212178:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212181:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212185:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212189:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212193:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212197:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212201:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212205:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212209:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212213:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212217:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212220:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212224:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212228:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212232:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212236:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212240:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212244:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212248:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212253:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212256:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212259:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212263:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212267:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212271:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212275:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212278:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212282:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212285:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212288:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212291:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212295:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212299:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212302:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212306:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212310:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212314:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212317:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212321:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212325:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212330:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212333:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212337:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212340:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212344:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212349:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212352:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212356:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212360:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212363:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212367:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212371:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212375:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212378:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212393:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212398:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212413:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212417:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212421:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212425:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212430:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212433:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212438:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212442:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212446:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212451:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212455:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212459:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212463:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212468:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212471:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212476:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212480:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212484:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212488:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212492:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212496:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212500:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212503:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212507:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212511:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212514:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212518:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212521:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212525:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212528:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212532:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212535:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212539:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212543:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212547:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212550:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212554:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212557:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212561:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212564:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212579:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212583:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212587:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212590:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212603:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212607:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212610:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212613:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212617:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212620:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212623:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212626:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212630:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212633:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212636:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212639:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200080:0.0:1788571993.212642:0:82773:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdfa043070 00020000:00000080:0.0:1788571993.212644:0:82773:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada517a2c00 uarg=0000000000000000 00000080:00200008:0.0:1788571993.212662:0:82773:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00010000:00010000:2.0:1788571993.979350:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571994.999715:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788571996.023377:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788571997.047235:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788571997.573974:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613632256:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571997.574102:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613632512:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571997.574141:0:2383:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1788571997.574148:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613632384:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571997.574173:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613632640:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571997.584165:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613632512:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571997.584217:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613632384:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571997.584233:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613632640:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788571997.584249:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613632256:192.168.202.113@tcp:400:kworker.0 00010000:00010000:2.0:1788571998.071763:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788571999.095163:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572000.133894:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572001.143075:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572002.167312:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572002.679664:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449613632768:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572002.687796:0:2383:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1788572002.687842:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449613632896:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572002.689841:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613633152:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572002.690010:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1788572002.690019:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613633024:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572002.690048:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613633280:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572002.694658:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449613632768:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572002.695410:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613633024:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572002.695423:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613633280:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572002.695435:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613633152:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572002.813181:0:82310:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f4a80 pname:cluuid:pid:xid:nid:opc:job multiop:35eb2d60-2de0-4b00-9815-d5153edd0057:82310:1875449613633536:192.168.202.113@tcp:35:multiop.0 00000002:00080000:3.0:1788572002.813216:0:82310:0:(mdc_request.c:1003:mdc_close()) @@@ Reset ESTALE = -116 req@ffff9ada603f4a80 x1875449613633536/t0(0) o35->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:23/10 lens 392/456 e 0 to 0 dl 1788572018 ref 1 fl Complete:RQU/600/0 rc -116/-116 job:'multiop.0' uid:0 gid:0 projid:0 00000002:00100000:3.0:1788572002.813256:0:82310:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9ada60a34000 x1875449613622528/t206158430211(206158430211) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/608 e 0 to 0 dl 1788571968 ref 1 fl Complete:RMQU/604/ffffffff rc 301/-1 job:'multiop.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1788572002.813311:0:82310:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x200017b01:0x2:0x0](ffff9ada58191988) 00010000:00010000:3.0:1788572003.191429:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572004.216405:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572005.241700:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572006.263093:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572007.289746:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572008.318170:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572009.349274:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572011.387399:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:0.0:1788572011.559679:0:83485:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == replay-single test 38: test recovery from unlink llog (test llog_gen_rec) ========================================================== 21:33:29 (1788572009) 00010000:00010000:3.0:1788572012.407795:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572012.922724:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516f2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613634688:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572012.939413:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516f2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449613634688:192.168.202.113@tcp:400:kworker.0 00010000:00010000:0.0:1788572013.436210:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572014.460151:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572031.868427:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572033.411465:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60742a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449614148480:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572033.411614:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60740380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614148736:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572033.411654:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60743100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614148992:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572033.411684:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1788572033.411690:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60742680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614148608:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572033.411727:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60741500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614148864:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572033.431725:0:2384:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 210453399104 gen 9 00000100:00100000:0.0:1788572033.439784:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60742680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614148608:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572033.439829:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60741500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614148864:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572033.439842:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60742a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449614148480:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572033.442020:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60740380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614148736:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572033.442057:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60743100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614148992:192.168.202.113@tcp:400:kworker.0 00010000:00010000:3.0:1788572043.127955:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572043.639150:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449614303360:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572043.639246:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449614303616:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572043.639281:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:3.0:1788572043.639287:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449614303232:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572043.639317:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449614303488:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572043.639336:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449614303744:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572043.644438:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449614303232:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572043.644462:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449614303488:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572043.644474:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449614303744:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572043.644487:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449614303360:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572043.644497:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449614303616:192.168.202.113@tcp:400:kworker.0 00000080:00200008:0.0:1788572043.988719:0:83888:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1788572043.988734:0:83888:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572043.988759:0:83888:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572043.988823:0:83888:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51637e00/0xef30c340bfb40d63 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 83888 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572043.988834:0:83888:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51637e00/0xef30c340bfb40d63 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 83888 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572043.988846:0:83888:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51637e00/0xef30c340bfb40d63 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 83888 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572043.988897:0:83888:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:83888:1875449614303872:192.168.202.113@tcp:101:lfs.0 00000100:00100000:0.0:1788572043.988991:0:83888:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6058bd80 going to sleep for 11 seconds 00000100:00100000:0.0:1788572044.016721:0:83888:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 210453399504 gen 9 00000100:00100000:0.0:1788572044.016884:0:83888:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:83888:1875449614303872:192.168.202.113@tcp:101:lfs.0 00010000:00010000:0.0:1788572044.016920:0:83888:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51637e00/0xef30c340bfb40d63 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306867 expref: -99 pid: 83888 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572044.016934:0:83888:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51637e00/0xef30c340bfb40d63 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306867 expref: -99 pid: 83888 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572044.017051:0:83888:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51637e00/0xef30c340bfb40d63 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8306867 expref: -99 pid: 83888 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572044.017065:0:83888:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51637e00/0xef30c340bfb40d63 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306867 expref: -99 pid: 83888 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572044.017081:0:83888:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51637e00/0xef30c340bfb40d63 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8306867 expref: -99 pid: 83888 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572044.017090:0:83888:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51637e00/0xef30c340bfb40d63 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8306867 expref: -99 pid: 83888 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572044.017098:0:83888:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51637e00/0xef30c340bfb40d63 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8306867 expref: -99 pid: 83888 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788572044.017112:0:83888:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada60626680 x1875449614303872/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788572059 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788572044.017126:0:83888:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51637e00/0xef30c340bfb40d63 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306867 expref: -99 pid: 83888 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572044.017139:0:83888:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1788572044.017146:0:83888:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1788572044.017177:0:83888:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 65536 00000080:00010000:0.0:1788572044.017183:0:83888:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb40d63 00000080:00010000:0.0:1788572044.017188:0:83888:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572044.017192:0:83888:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb40d63 from it ffffab0ec1c77cb8 00010000:00010000:0.0:1788572044.017196:0:83888:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51637e00/0xef30c340bfb40d63 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8306867 expref: -99 pid: 83888 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572044.017205:0:83888:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51637e00/0xef30c340bfb40d63 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306867 expref: -99 pid: 83888 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:0.0:1788572044.018345:0:83888:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1788572044.018354:0:83888:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572044.018375:0:83888:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572044.018391:0:83888:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51637e00/0xef30c340bfb40d63 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306867 expref: -99 pid: 83888 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788572044.018404:0:83888:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51637e00/0xef30c340bfb40d63 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306867 expref: -99 pid: 83888 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572044.018412:0:83888:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572044.018418:0:83888:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572044.018422:0:83888:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb40d63 00000080:00010000:0.0:1788572044.018427:0:83888:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb40d63 from it ffffab0ec1c77cb8 00010000:00010000:0.0:1788572044.018431:0:83888:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51637e00/0xef30c340bfb40d63 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8306867 expref: -99 pid: 83888 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788572044.018440:0:83888:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51637e00/0xef30c340bfb40d63 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306867 expref: -99 pid: 83888 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788572044.022303:0:83888:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788572044.022348:0:83888:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788572044.022370:0:83888:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51637e00/0xef30c340bfb40d63 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306867 expref: -99 pid: 83888 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788572044.022388:0:83888:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51637e00/0xef30c340bfb40d63 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306867 expref: -99 pid: 83888 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788572044.022397:0:83888:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788572044.022405:0:83888:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788572044.022409:0:83888:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb40d63 00000080:00010000:2.0:1788572044.022416:0:83888:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb40d63 from it ffffab0ec1c77bb0 00010000:00010000:2.0:1788572044.022420:0:83888:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51637e00/0xef30c340bfb40d63 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8306867 expref: -99 pid: 83888 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788572044.022435:0:83888:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51637e00/0xef30c340bfb40d63 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306867 expref: -99 pid: 83888 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00000080:00200000:2.0:1788572044.022451:0:83888:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:2.0:1788572044.022461:0:83888:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200080:2.0:1788572044.022588:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:2.0:1788572044.022605:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454f3800 uarg=0000000000000000 00800000:00000004:2.0:1788572044.022613:0:83888:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: age 3910, max_age 3960 00000100:00100000:2.0:1788572044.022659:0:83888:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:83888:1875449614304000:192.168.202.113@tcp:41:lfs.0 00000100:00100000:2.0:1788572044.022754:0:83888:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada605c2180 going to sleep for 11 seconds 00000100:00100000:3.0:1788572044.042223:0:83888:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:83888:1875449614304000:192.168.202.113@tcp:41:lfs.0 00800000:00000004:3.0:1788572044.042257:0:83888:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: update ffff9ada4481d198 cache blocks 317318/353529 objects 1023288/1024000 00000080:00200080:3.0:1788572044.042355:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.042385:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00800000:00000004:3.0:1788572044.042396:0:83888:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: age 3910, max_age 3960 00000100:00100000:3.0:1788572044.042436:0:83888:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c3800 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:83888:1875449614304128:192.168.202.113@tcp:41:lfs.0 00000100:00100000:3.0:1788572044.042540:0:83888:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada50f7a9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788572044.053555:0:83888:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c3800 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:83888:1875449614304128:192.168.202.113@tcp:41:lfs.0 00800000:00000004:3.0:1788572044.053598:0:83888:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: update ffff9ada44819e18 cache blocks 317392/353529 objects 1023736/1024000 00000080:00200080:3.0:1788572044.053683:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.053702:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.053710:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.053714:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.053718:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.053721:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.053725:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.053728:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.053732:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.053736:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.053741:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.053745:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.053749:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.053752:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.053756:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.053759:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.053763:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.053768:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.053771:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.053775:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.053779:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.053782:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.053786:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.053789:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.053792:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.053795:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.053799:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.053803:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.053806:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.053810:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.053814:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.053817:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.053821:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.053824:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.053828:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.053832:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.053836:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.053840:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.053844:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.053847:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.053850:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.053891:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.053897:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.053901:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.053904:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.053908:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.053913:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.053917:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.053921:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.053924:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.053928:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.053932:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.053937:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.053940:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.053945:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.053949:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.053953:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.053957:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.053962:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.053966:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.053970:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.053974:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.053979:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.053982:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.054001:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.054006:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.054011:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.054016:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.054020:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.054024:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.054028:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.054032:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.054036:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.054041:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.054044:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.054048:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.054052:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.054056:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.054061:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.054064:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.054069:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.054072:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.054076:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.054080:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.054085:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.054089:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.054093:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.054097:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.054101:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.054105:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.054109:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.054113:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.054118:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.054122:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.054126:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.054130:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.054135:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.054139:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.054150:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.054155:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.054159:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.054163:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.054168:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.054172:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.054177:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.054191:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.054196:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.054200:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.054204:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.054208:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.054212:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.054216:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.054221:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.054225:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.054229:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.054233:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.054237:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.054242:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.054246:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.054250:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.054254:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.054258:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.054263:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.054268:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.054271:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00800000:00000080:3.0:1788572044.054275:0:83888:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.054280:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.054287:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00020000:00000004:3.0:1788572044.054296:0:83888:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: age 3910, max_age 3960 00000100:00100000:3.0:1788572044.054347:0:83888:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c3800 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:83888:1875449614304256:192.168.202.113@tcp:13:lfs.0 00000100:00100000:3.0:1788572044.054435:0:83888:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada50f7a9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788572044.062125:0:83888:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c3800 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:83888:1875449614304256:192.168.202.113@tcp:13:lfs.0 00020000:00000004:3.0:1788572044.062157:0:83888:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: update ffff9ada4354d198 cache blocks 901357/958279 objects 261510/262144 00000080:00200080:3.0:1788572044.063142:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.063167:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00020000:00000004:3.0:1788572044.063252:0:83888:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: age 3910, max_age 3960 00000100:00100000:3.0:1788572044.063302:0:83888:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c3800 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:83888:1875449614304384:192.168.202.113@tcp:13:lfs.0 00000100:00100000:3.0:1788572044.063767:0:83888:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada50f7a9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788572044.078846:0:83888:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c3800 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:83888:1875449614304384:192.168.202.113@tcp:13:lfs.0 00020000:00000004:3.0:1788572044.078891:0:83888:0:(obd_class.h:1116:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: update ffff9ada43549e18 cache blocks 901363/958279 objects 261518/262144 00000080:00200080:3.0:1788572044.079362:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079383:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079392:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079398:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079402:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079405:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079409:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079413:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079417:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079420:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079424:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079428:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079432:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079435:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079440:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079443:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079448:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079451:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079455:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079459:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079463:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079467:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079471:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079475:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079480:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079484:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079488:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079492:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079497:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079500:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079505:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079509:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079513:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079517:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079521:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079524:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079529:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079532:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079537:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079540:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079544:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079547:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079551:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079555:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079559:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079562:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079566:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079570:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079574:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079578:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079582:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079587:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079590:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079594:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079597:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079600:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079604:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079607:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079611:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079614:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079618:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079621:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079625:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079629:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079633:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079636:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079640:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079643:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079646:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079650:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079653:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079657:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079660:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079664:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079668:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079672:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079676:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079680:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079684:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079688:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079692:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079695:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079700:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079703:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079707:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079712:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079715:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079719:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079723:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079726:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079730:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079734:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079738:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079742:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079747:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079750:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079754:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079758:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079763:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079767:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079771:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079776:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079780:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079784:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079787:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079791:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079795:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079798:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079802:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079805:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079809:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079812:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079815:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079818:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079822:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079825:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079829:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079832:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079836:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079840:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079844:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079847:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079851:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079884:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200080:3.0:1788572044.079889:0:83888:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff02e76c60 00020000:00000080:3.0:1788572044.079893:0:83888:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165c400 uarg=0000000000000000 00000080:00200008:3.0:1788572044.079908:0:83888:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200000:0.0:1788572044.154989:0:83895:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572044.155022:0:83895:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572044.155044:0:83895:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51637e00/0xef30c340bfb40d63 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306867 expref: -99 pid: 83888 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00010000:00010000:0.0:1788572044.155060:0:83895:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51637e00/0xef30c340bfb40d63 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306867 expref: -99 pid: 83888 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00000080:00200000:0.0:1788572044.155068:0:83895:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572044.155076:0:83895:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572044.155079:0:83895:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb40d63 00000080:00010000:0.0:1788572044.155087:0:83895:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb40d63 from it ffffab0ec19dfbd0 00010000:00010000:0.0:1788572044.155090:0:83895:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51637e00/0xef30c340bfb40d63 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8306867 expref: -99 pid: 83888 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00010000:00010000:0.0:1788572044.155099:0:83895:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51637e00/0xef30c340bfb40d63 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306867 expref: -99 pid: 83888 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00200000:0.0:1788572044.155113:0:83895:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:0.0:1788572044.155127:0:83895:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=fsa-oleg213-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), flags=1536 00000080:00200000:0.0:1788572044.155131:0:83895:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572044.155137:0:83895:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572044.155143:0:83895:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51637e00/0xef30c340bfb40d63 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306867 expref: -99 pid: 83888 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:0.0:1788572044.155150:0:83895:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51637e00/0xef30c340bfb40d63 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306867 expref: -99 pid: 83888 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00200000:0.0:1788572044.155156:0:83895:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572044.155160:0:83895:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572044.155162:0:83895:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb40d63 00000080:00010000:0.0:1788572044.155167:0:83895:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb40d63 from it ffffab0ec19dfcd0 00010000:00010000:0.0:1788572044.155170:0:83895:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51637e00/0xef30c340bfb40d63 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8306867 expref: -99 pid: 83888 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:0.0:1788572044.155180:0:83895:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51637e00/0xef30c340bfb40d63 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306867 expref: -99 pid: 83888 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:0.0:1788572044.155187:0:83895:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:0.0:1788572044.155192:0:83895:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572044.155198:0:83895:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572044.155203:0:83895:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51637e00/0xef30c340bfb40d63 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306867 expref: -99 pid: 83888 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:0.0:1788572044.155210:0:83895:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51637e00/0xef30c340bfb40d63 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306867 expref: -99 pid: 83888 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:0.0:1788572044.155215:0:83895:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572044.155218:0:83895:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572044.155220:0:83895:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb40d63 00000080:00010000:0.0:1788572044.155223:0:83895:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb40d63 from it ffffab0ec19dfdb0 00010000:00010000:0.0:1788572044.155226:0:83895:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51637e00/0xef30c340bfb40d63 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8306867 expref: -99 pid: 83888 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:0.0:1788572044.155232:0:83895:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51637e00/0xef30c340bfb40d63 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306867 expref: -99 pid: 83888 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000080:00200000:0.0:1788572044.155238:0:83895:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:0.0:1788572044.155244:0:83895:0:(namei.c:2062:ll_create_nd()) VFS Op:name=fsa-oleg213-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), flags=33188, excl=1 00000080:00200000:0.0:1788572044.155248:0:83895:0:(namei.c:2017:ll_mknod()) VFS Op:name=fsa-oleg213-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9ada58125b88) mode 100644 dev 0 00010000:00010000:0.0:1788572044.155273:0:83895:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51637e00/0xef30c340bfb40d63 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b8306867 expref: -99 pid: 83888 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:0.0:1788572044.155389:0:83895:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51637e00/0xef30c340bfb40d63 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b8306867 expref: -99 pid: 83888 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:0.0:1788572044.155397:0:83895:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1788572044.155400:0:83895:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51637e00/0xef30c340bfb40d63 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b8306867 expref: -99 pid: 83888 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000020:00100000:0.0:1788572044.155425:0:83895:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1788572044.155438:0:83895:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada60626680 x1875449614304640/t0(0) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788572044.155465:0:83895:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job mcreate:35eb2d60-2de0-4b00-9815-d5153edd0057:83895:1875449614304640:192.168.202.113@tcp:36:mcreate.0 00000100:00100000:0.0:1788572044.155555:0:83895:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6058bd80 going to sleep for 11 seconds 00000100:00100000:0.0:1788572044.183215:0:83895:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job mcreate:35eb2d60-2de0-4b00-9815-d5153edd0057:83895:1875449614304640:192.168.202.113@tcp:36:mcreate.0 00000080:00200000:0.0:1788572044.183275:0:83895:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x2000182d1:0x321:0x0](ffff9ada4626cb08) 00000080:00200000:0.0:1788572044.183319:0:83895:0:(namei.c:2070:ll_create_nd()) VFS Op:name=fsa-oleg213-client.virtnet, unhashed 1 00000080:00200000:3.0:1788572044.205488:0:83898:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:3.0:1788572044.205542:0:83898:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1788572044.205623:0:83898:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45518a00/0xef30c340bfb40d6a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 83898 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572044.205635:0:83898:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45518a00/0xef30c340bfb40d6a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 83898 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572044.205646:0:83898:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45518a00/0xef30c340bfb40d6a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 83898 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572044.205674:0:83898:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c3800 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:83898:1875449614304768:192.168.202.113@tcp:101:rm.0 00000100:00100000:3.0:1788572044.207252:0:83898:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada50f7a840 going to sleep for 11 seconds 00000100:00100000:0.0:1788572044.208904:0:83898:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c3800 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:83898:1875449614304768:192.168.202.113@tcp:101:rm.0 00010000:00010000:0.0:1788572044.208932:0:83898:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45518a00/0xef30c340bfb40d6a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306883 expref: -99 pid: 83898 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572044.208946:0:83898:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45518a00/0xef30c340bfb40d6a lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306883 expref: -99 pid: 83898 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572044.208959:0:83898:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45518a00/0xef30c340bfb40d6a lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8306883 expref: -99 pid: 83898 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572044.208972:0:83898:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45518a00/0xef30c340bfb40d6a lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306883 expref: -99 pid: 83898 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572044.208986:0:83898:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45518a00/0xef30c340bfb40d6a lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8306883 expref: -99 pid: 83898 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572044.208995:0:83898:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45518a00/0xef30c340bfb40d6a lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8306883 expref: -99 pid: 83898 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572044.209003:0:83898:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45518a00/0xef30c340bfb40d6a lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8306883 expref: -99 pid: 83898 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788572044.209015:0:83898:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada454c3800 x1875449614304768/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788572060 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788572044.209029:0:83898:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45518a00/0xef30c340bfb40d6a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306883 expref: -99 pid: 83898 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572044.209043:0:83898:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1788572044.209048:0:83898:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00010000:00010000:3.0:1788572046.199425:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572048.759439:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449614305408:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572048.759546:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449614305664:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572048.759584:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:3.0:1788572048.759591:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449614305280:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572048.759620:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449614305536:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572048.759644:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449614305792:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572048.783983:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449614305664:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572048.784033:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449614305280:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572048.784048:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449614305536:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572048.784062:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449614305792:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572048.784078:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449614305408:192.168.202.113@tcp:400:kworker.0 00010000:00010000:3.0:1788572049.275057:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572052.343298:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1788572053.258251:0:83965:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: local REPLAY BARRIER on lustre-MDT0000 00000100:00100000:0.0:1788572053.879460:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614306048:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572053.879823:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614306304:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572053.917504:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614306048:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572053.917529:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614306304:192.168.202.113@tcp:400:kworker.0 00010000:00010000:3.0:1788572055.415174:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572058.516042:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572059.002274:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449614306816:192.168.202.113@tcp:400:kworker.0 00000100:00080000:0.0:1788572059.002297:0:2383:0:(recover.c:205:ptlrpc_request_handle_notconn()) import lustre-MDT0000-mdc-ffff9ada58b0a000 of lustre-MDT0000_UUID@192.168.202.113@tcp abruptly disconnected: reconnecting 00000100:00080000:0.0:1788572059.002304:0:2383:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:0.0:1788572059.002308:0:2383:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff9ada58b0a000: Connection to lustre-MDT0000 (at 192.168.202.113@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:0.0:1788572059.002315:0:2383:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572059.002320:0:2383:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 3899 00000100:00080000:0.0:1788572059.002332:0:2383:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788572059.002344:0:2383:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788572059.002386:0:2383:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449614307200/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788572059.002422:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449614306688:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572059.003632:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449614307200:192.168.202.113@tcp:38:ptlrpcd_00_00.0 00000100:00080000:2.0:1788572059.014988:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788572059.015014:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-19) 00000100:00100000:2.0:1788572059.015021:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449614307200:192.168.202.113@tcp:38:ptlrpcd_00_00.0 00000100:00100000:0.0:1788572059.016696:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449614307072:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572059.016718:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449614306944:192.168.202.113@tcp:400:kworker.0 00010000:00010000:0.0:1788572059.511934:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572061.561933:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572062.584173:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572064.119133:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449614307328:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572064.119260:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449614307456:192.168.202.113@tcp:38:kworker.0 00000100:00080000:0.0:1788572064.129592:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788572064.129600:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-19) 00000100:00100000:0.0:1788572064.129605:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449614307456:192.168.202.113@tcp:38:kworker.0 00010000:00010000:3.0:1788572064.631200:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572065.660443:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572067.703889:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572068.728129:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788572069.246205:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572069.246213:0:58077:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788572069.246216:0:58077:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572069.246227:0:58077:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 3981 00000100:00080000:3.0:1788572069.246239:0:58077:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 10s 00000100:00080000:3.0:1788572069.246247:0:58077:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572069.246259:0:58077:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572069.246277:0:58077:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada454c1f80 x1875449614308096/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788572069.246346:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449614307968:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572069.246433:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449614308352:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572069.246464:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1788572069.246470:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449614308224:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572069.246493:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449614308480:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572069.246526:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449614308096:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572069.270344:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449614308480:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572069.270365:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449614308352:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572069.271629:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449614308224:192.168.202.113@tcp:400:kworker.0 00010000:00010000:3.0:1788572070.782050:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572071.799355:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572073.847387:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000400:0.0:1788572074.359914:0:2383:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788572058/real 1788572058] req@ffff9ada455e0700 x1875449614306560/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 1 dl 1788572074 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572074.359953:0:2383:0:(client.c:2548:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9ada455e0700 x1875449614306560/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 1 dl 1788572074 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572074.359963:0:2383:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from FULL to DISCONN 00000100:02020000:0.0:1788572074.359968:0:2383:0:(import.c:186:ptlrpc_set_import_discon()) MGC192.168.202.113@tcp: Connection to MGS (at 192.168.202.113@tcp) was lost; in progress operations using this service will fail 00000100:00080000:0.0:1788572074.359971:0:2383:0:(import.c:192:ptlrpc_set_import_discon()) import MGS@192.168.202.113@tcp for MGC192.168.202.113@tcp not replayable, auto-deactivating 00000100:00080000:0.0:1788572074.359974:0:2383:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import MGS INVALID 00000100:00100000:0.0:1788572074.359978:0:2383:0:(client.c:3591:ptlrpc_abort_inflight()) @@@ inflight req@ffff9ada455e0700 x1875449614306560/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 1 dl 1788572074 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572074.359988:0:2383:0:(client.c:3591:ptlrpc_abort_inflight()) @@@ inflight req@ffff9ada455e2a00 x1875449614307328/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572080 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572074.359998:0:2383:0:(client.c:3591:ptlrpc_abort_inflight()) @@@ inflight req@ffff9ada454c0a80 x1875449614307968/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572085 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:0.0:1788572074.360021:0:2383:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808001 00000100:00080000:0.0:1788572074.360024:0:2383:0:(pinger.c:193:ptlrpc_pinger_ir_down()) IR down 10000000:01000000:0.0:1788572074.360026:0:2383:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808002 00000100:00080000:0.0:1788572074.360028:0:2383:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00100000:0.0:1788572074.360068:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449614307328:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572074.360076:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449614306560:192.168.202.113@tcp:400:kworker.0 00000100:00080000:0.0:1788572074.360115:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572074.360131:0:58077:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788572074.360135:0:58077:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572074.360141:0:58077:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 3899 00000100:00080000:0.0:1788572074.360158:0:58077:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788572074.360194:0:58077:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627100 x1875449614308608/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788572074.360222:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572074.360227:0:58077:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788572074.360229:0:58077:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572074.360233:0:58077:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 3986 00000100:00080000:0.0:1788572074.360238:0:58077:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 15s 00000100:00080000:0.0:1788572074.360251:0:58077:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788572074.360262:0:58077:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788572074.360274:0:58077:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449614308736/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788572074.360331:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449614308864:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572074.360413:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449614309120:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572074.360443:0:2383:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1788572074.360449:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449614308992:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572074.360496:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449614308608:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788572074.360525:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449614308736:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572074.361816:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449614307968:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572074.365737:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449614308992:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572074.365756:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449614308864:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572074.365770:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449614309120:192.168.202.113@tcp:400:kworker.0 00000100:00080000:0.0:1788572074.365816:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788572074.365823:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-19) 00000100:00100000:0.0:1788572074.365828:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449614308736:192.168.202.113@tcp:38:kworker.0 00010000:00010000:0.0:1788572074.871344:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572077.949572:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572079.483215:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449614309248:192.168.202.113@tcp:38:kworker.0 00000100:00080000:0.0:1788572079.498206:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788572079.498222:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-19) 00000100:00100000:0.0:1788572079.498228:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449614309248:192.168.202.113@tcp:38:kworker.0 00010000:00010000:0.0:1788572081.015923:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572084.087230:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572084.599743:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788572074/real 1788572074] req@ffff9ada60627100 x1875449614308608/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572084 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572084.599869:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627100 x1875449614308608/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572084 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572084.599885:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449614309760:192.168.202.113@tcp:38:kworker.0 00000100:00080000:3.0:1788572084.599934:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572084.599954:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572084.599958:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449614308608:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572084.600257:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449614310272:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788572084.621000:0:2382:0:(import.c:1176:ptlrpc_connect_interpret()) MGC192.168.202.113@tcp: connect to target with instance 0 10000000:01000000:3.0:1788572084.621025:0:2382:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808005 00000100:02000400:3.0:1788572084.621036:0:2382:0:(import.c:1328:ptlrpc_connect_interpret()) Evicted from MGS (at 192.168.202.113@tcp) after server handle changed from 0xdb7edb52b82f81a6 to 0xdb7edb52b83068de 00000100:00080000:3.0:1788572084.621042:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to EVICTED 00000100:00080000:3.0:1788572084.621049:0:2382:0:(import.c:1718:ptlrpc_import_recovery_state_machine()) evicted from MGS@192.168.202.113@tcp; invalidating 00000100:00100000:2.0:1788572084.621687:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449614310272:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788572084.622744:0:84187:0:(import.c:1642:ptlrpc_invalidate_import_thread()) thread invalidate import MGC192.168.202.113@tcp to MGS@192.168.202.113@tcp 00000100:00100000:3.0:1788572084.622751:0:84187:0:(import.c:323:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 10000000:01000000:3.0:1788572084.622757:0:84187:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808003 00010000:00010000:3.0:1788572084.622782:0:84187:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.202.113@tcp lock: ffff9ada53017200/0xef30c340bfb3d69e lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x110e400000000 nid: local remote: 0xdb7edb52b82f8278 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1788572084.622797:0:84187:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.202.113@tcp lock: ffff9ada53017200/0xef30c340bfb3d69e lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x110f400000000 nid: local remote: 0xdb7edb52b82f8278 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1788572084.622803:0:84187:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:3.0:1788572084.622828:0:84187:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1788572084.622840:0:84187:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.202.113@tcp lock: ffff9ada53017200/0xef30c340bfb3d69e lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x590f400000000 nid: local remote: 0xdb7edb52b82f8278 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572084.622898:0:84187:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.202.113@tcp lock: ffff9ada516e0800/0xef30c340bfb3d6ac lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0xdb7edb52b82f8286 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1788572084.622905:0:84187:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.202.113@tcp lock: ffff9ada516e0800/0xef30c340bfb3d6ac lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0xdb7edb52b82f8286 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1788572084.622911:0:84187:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:3.0:1788572084.622915:0:84187:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1788572084.622919:0:84187:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.202.113@tcp lock: ffff9ada516e0800/0xef30c340bfb3d6ac lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0xdb7edb52b82f8286 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572084.622935:0:84187:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.202.113@tcp lock: ffff9ada516e1600/0xef30c340bfb3d697 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0xdb7edb52b82f8271 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1788572084.622942:0:84187:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.202.113@tcp lock: ffff9ada516e1600/0xef30c340bfb3d697 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0xdb7edb52b82f8271 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1788572084.622947:0:84187:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:3.0:1788572084.622951:0:84187:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1788572084.622954:0:84187:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.202.113@tcp lock: ffff9ada516e1600/0xef30c340bfb3d697 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0xdb7edb52b82f8271 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572084.622966:0:84187:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.202.113@tcp lock: ffff9ada516e1c00/0xef30c340bfb3d6a5 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0xdb7edb52b82f827f expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1788572084.622975:0:84187:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.202.113@tcp lock: ffff9ada516e1c00/0xef30c340bfb3d6a5 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0xdb7edb52b82f827f expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1788572084.622979:0:84187:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:3.0:1788572084.622982:0:84187:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1788572084.622985:0:84187:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.202.113@tcp lock: ffff9ada516e1c00/0xef30c340bfb3d6a5 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0xdb7edb52b82f827f expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1788572084.622998:0:84187:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from EVICTED to RECOVER 00000100:00080000:3.0:1788572084.623004:0:84187:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from RECOVER to FULL 10000000:01000000:3.0:1788572084.623006:0:84187:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808004 00000100:00080000:3.0:1788572084.623009:0:84187:0:(pinger.c:186:ptlrpc_pinger_ir_up()) IR up 00000100:02000000:3.0:1788572084.623013:0:84187:0:(import.c:1787:ptlrpc_import_recovery_state_machine()) MGC192.168.202.113@tcp: Connection restored to 192.168.202.113@tcp (at 192.168.202.113@tcp) 00000100:00100000:3.0:1788572085.112199:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449614310400:192.168.202.113@tcp:17:kworker.0 00000100:00100000:3.0:1788572085.128946:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449614310400:192.168.202.113@tcp:17:kworker.0 00000100:00100000:3.0:1788572086.102639:0:15207:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e0000 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449614310656:192.168.202.113@tcp:256:ll_cfg_requeue.0 00000100:00080000:3.0:1788572086.102742:0:15207:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff9ada51140000 already not connected (conn 333, was 0): DISCONN 00010000:00080000:3.0:1788572086.102760:0:15207:0:(ldlm_lib.c:85:import_set_conn()) imp ffff9ada51140000@lustre-MDT0000-mdc-ffff9ada58b0a000: found existing conn 192.168.202.113@tcp, moved to head 00000100:00080000:3.0:1788572086.102767:0:15207:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572086.102773:0:15207:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 0 00000100:00080000:3.0:1788572086.102783:0:15207:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572086.102797:0:15207:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572086.102845:0:15207:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada454c0a80 x1875449614310784/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 10000000:01000000:3.0:1788572086.102963:0:15207:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.202.113@tcp: configuration from log 'lustre-cliir' succeeded (0). 00010000:00010000:3.0:1788572086.102975:0:15207:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.202.113@tcp lock: ffff9ada607bd000/0xef30c340bfb40d78 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b8306a04 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572086.102986:0:15207:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.202.113@tcp lock: ffff9ada607bd000/0xef30c340bfb40d78 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b8306a04 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1788572086.102997:0:15207:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:3.0:1788572086.103000:0:15207:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9ada58b0a000 from 52 10000000:01000000:3.0:1788572086.103003:0:15207:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:3.0:1788572086.103036:0:15207:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.202.113@tcp lock: ffff9ada45519e00/0xef30c340bfb40d7f lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572086.103044:0:15207:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.202.113@tcp lock: ffff9ada45519e00/0xef30c340bfb40d7f lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572086.103055:0:15207:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.202.113@tcp lock: ffff9ada45519e00/0xef30c340bfb40d7f lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572086.103076:0:15207:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c0000 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449614310912:192.168.202.113@tcp:101:ll_cfg_requeue.0 00000100:00100000:3.0:1788572086.103163:0:15207:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada51982000 going to sleep for 11 seconds 00000100:00100000:2.0:1788572086.104911:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449614310784:192.168.202.113@tcp:38:ll_cfg_requeue.0 00000100:00100000:3.0:1788572086.142510:0:15207:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e0a80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449614311040:192.168.202.113@tcp:501:ll_cfg_requeue.0 00000100:00100000:3.0:1788572086.142594:0:15207:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c1c00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449614311168:192.168.202.113@tcp:503:ll_cfg_requeue.0 00000100:00100000:3.0:1788572086.142732:0:15207:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada51982480 going to sleep for 11 seconds 00000100:00100000:3.0:1788572086.158240:0:15207:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c1c00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449614311168:192.168.202.113@tcp:503:ll_cfg_requeue.0 00000040:00080000:3.0:1788572086.160018:0:84198:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 64768 count 51 00000040:00080000:2.0:1788572086.190606:0:84199:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x2:0x0] index 64768 count 7 00000100:00100000:3.0:1788572086.212131:0:15207:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e1c00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449614311808:192.168.202.113@tcp:501:ll_cfg_requeue.0 10000000:01000000:3.0:1788572086.212198:0:15207:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.202.113@tcp: configuration from log 'lustre-sptlrpc' failed (-2). 00010000:00010000:3.0:1788572086.212212:0:15207:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.202.113@tcp lock: ffff9ada607bd400/0xef30c340bfb40d8d lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b8306a7b expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572086.212225:0:15207:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.202.113@tcp lock: ffff9ada607bd400/0xef30c340bfb40d8d lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b8306a7b expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572087.160893:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572089.720451:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449614311936:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572089.720596:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614312192:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572089.720634:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1788572089.720641:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614312064:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572089.720665:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614312320:192.168.202.113@tcp:400:kworker.0 00010000:00010000:0.0:1788572089.730976:0:2384:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 0/0 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1788572089.731001:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614312064:192.168.202.113@tcp:400:kworker.0 00010000:00010000:0.0:1788572089.731035:0:2384:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 0/0 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1788572089.731041:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614312320:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572089.731054:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449614311936:192.168.202.113@tcp:400:kworker.0 00010000:00010000:0.0:1788572089.731202:0:2384:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 0/0 locks to regular blp list, 3 blwis in pool 00000100:00100000:0.0:1788572089.731210:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614312192:192.168.202.113@tcp:400:kworker.0 00010000:00010000:3.0:1788572089.741370:0:15205:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 0 locks of 0 total in blp. (2 blwis in pool) 00010000:00010000:3.0:1788572089.741406:0:15205:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 0 locks of 0 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1788572089.741413:0:15205:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 0 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1788572089.741419:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572089.741421:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572089.741423:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572089.742270:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572090.238504:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788572090.388533:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449614305152:192.168.202.113@tcp:36:rm.0 00010000:00010000:1.0:1788572090.413733:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4548a800/0xef30c340bfb3f7ec lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x260:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82ff702 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.413751:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x25f:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.413762:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4548be00/0xef30c340bfb3f7de lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x25f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82ff6d1 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.413776:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x25e:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.413788:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4548aa00/0xef30c340bfb3f7d0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x25e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82ff6a0 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.413802:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x25d:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.413828:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4548b000/0xef30c340bfb3f7c2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x25d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82ff66f expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.414005:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x25c:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.414020:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4548ae00/0xef30c340bfb3f7b4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x25c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82ff63e expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.414034:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x25b:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.414046:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51945200/0xef30c340bfb3f7a6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x25b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82ff60d expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.414062:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x25a:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.414072:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51945e00/0xef30c340bfb3f798 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x25a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82ff5dc expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.414087:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x259:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.414098:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51944400/0xef30c340bfb3f78a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x259:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82ff5ab expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.414116:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x258:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.414128:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4548ba00/0xef30c340bfb3f77c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x258:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82ff57a expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.414143:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x257:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.414154:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4548bc00/0xef30c340bfb3f76e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x257:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82ff549 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.414176:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x256:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.414190:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4548b800/0xef30c340bfb3f760 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x256:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82ff518 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.414207:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x255:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.414219:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4548b400/0xef30c340bfb3f752 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x255:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82ff4e7 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.414234:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x254:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.414245:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e9e00/0xef30c340bfb3f744 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x254:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82ff4b6 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.414270:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x253:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.414284:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51944800/0xef30c340bfb3f736 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x253:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82ff485 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.414299:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x252:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.414309:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51945000/0xef30c340bfb3f728 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x252:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82ff454 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.414331:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x251:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.414646:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4548b200/0xef30c340bfb3f71a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x251:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82ff423 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.414669:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x250:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.414680:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50b61400/0xef30c340bfb3f70c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x250:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82ff3f2 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.414695:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x24f:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.414707:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada604f8e00/0xef30c340bfb3f6fe lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x24f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82ff3c1 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.414721:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x24e:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.414734:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51945800/0xef30c340bfb3f6f0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x24e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82ff390 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.414749:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x24d:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.414760:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51944a00/0xef30c340bfb3f6e2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x24d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82ff35f expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.414777:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x24c:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.414790:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51945600/0xef30c340bfb3f6d4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x24c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82ff32e expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.414805:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x24b:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.414829:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51945c00/0xef30c340bfb3f6c6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x24b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82ff2fd expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.415098:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x24a:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.415115:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51944e00/0xef30c340bfb3f6b8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x24a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82ff2cc expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.415130:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x249:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.415140:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada604f8800/0xef30c340bfb3f6aa lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x249:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82ff29b expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.415157:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x248:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.415170:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada604f8c00/0xef30c340bfb3f69c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x248:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82ff26a expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.415187:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x247:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.415200:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada604f9800/0xef30c340bfb3f68e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x247:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82ff239 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.415217:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x246:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.415230:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada604f9200/0xef30c340bfb3f680 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x246:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82ff208 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.415244:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x245:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.415257:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51945400/0xef30c340bfb3f672 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x245:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82ff1d7 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.415273:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x244:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.415284:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada604f9400/0xef30c340bfb3f664 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x244:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82ff1a6 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.415303:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x243:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.415315:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada604f9a00/0xef30c340bfb3f656 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x243:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82ff175 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.415405:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x242:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.415424:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada604f8200/0xef30c340bfb3f648 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x242:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82ff144 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.415441:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x241:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.415453:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51944600/0xef30c340bfb3f63a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x241:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82ff113 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.415466:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x240:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.415477:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51944c00/0xef30c340bfb3f62c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x240:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82ff0e2 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.415504:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x23f:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.415517:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51944000/0xef30c340bfb3f61e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x23f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82ff0b1 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.415531:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x23e:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.415621:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51944200/0xef30c340bfb3f610 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x23e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82ff080 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.415642:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x23d:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.415656:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45425e00/0xef30c340bfb3f602 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x23d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82ff04f expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.415676:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x23c:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.415689:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606cb200/0xef30c340bfb3f5f4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x23c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82ff01e expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.415705:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x23b:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.415722:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8c00/0xef30c340bfb3f5e6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x23b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fefed expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.415738:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x23a:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.416688:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada603e8000/0xef30c340bfb3f5d8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x23a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fefbc expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.416724:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x239:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.416739:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17800/0xef30c340bfb3f5ca lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x239:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fef8b expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.416756:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x238:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.416769:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606ca000/0xef30c340bfb3f5bc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x238:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fef5a expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.416786:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x237:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.416799:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606ca800/0xef30c340bfb3f5ae lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x237:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fef29 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.416813:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x236:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.416826:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606cac00/0xef30c340bfb3f5a0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x236:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82feef8 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.416882:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x235:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.416897:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada604f8600/0xef30c340bfb3f592 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x235:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82feec7 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.416914:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x234:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.416928:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada604f8000/0xef30c340bfb3f584 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x234:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fee96 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.416943:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x233:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.416955:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606ca200/0xef30c340bfb3f576 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x233:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fee65 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.416971:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x232:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.416984:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606ca400/0xef30c340bfb3f568 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x232:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fee34 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.416999:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x231:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.417011:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606caa00/0xef30c340bfb3f55a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x231:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fee03 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.417028:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x230:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.417041:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada604f9600/0xef30c340bfb3f54c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x230:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fedd2 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.417057:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x22f:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.417069:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada604f8400/0xef30c340bfb3f53e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x22f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82feda1 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.417091:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x22e:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.417104:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606cbe00/0xef30c340bfb3f530 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x22e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fed70 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.417119:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x22d:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.417131:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606cb600/0xef30c340bfb3f522 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x22d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fed3f expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.417146:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x22c:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.417159:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606cb000/0xef30c340bfb3f514 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x22c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fed0e expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.417174:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x22b:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.417187:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606cb400/0xef30c340bfb3f506 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x22b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fecdd expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.417202:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x22a:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.417212:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada604f9e00/0xef30c340bfb3f4f8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x22a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fecac expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.417226:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x229:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.417238:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada604f8a00/0xef30c340bfb3f4ea lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x229:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fec7b expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.417254:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x228:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.417267:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606cb800/0xef30c340bfb3f4dc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x228:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fec4a expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.417287:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x227:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.417298:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606cbc00/0xef30c340bfb3f4ce lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x227:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fec19 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.417314:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x226:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.417325:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606cae00/0xef30c340bfb3f4c0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x226:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82febe8 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.417491:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x225:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.417510:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606cba00/0xef30c340bfb3f4b2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x225:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82febb7 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.417526:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x224:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.417538:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45425600/0xef30c340bfb3f4a4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x224:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82feb86 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.417628:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x223:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.417643:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45424a00/0xef30c340bfb3f496 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x223:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82feb55 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.417661:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x222:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.417675:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45425000/0xef30c340bfb3f488 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x222:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82feb24 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.417691:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x221:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.417706:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45425c00/0xef30c340bfb3f47a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x221:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82feaf3 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.417730:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x220:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.417742:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45424600/0xef30c340bfb3f46c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x220:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82feac2 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.417758:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x21f:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.417769:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45424e00/0xef30c340bfb3f45e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x21f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fea91 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.417784:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x21e:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.417795:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45425800/0xef30c340bfb3f450 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x21e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fea60 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.417810:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x21d:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.417822:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45425400/0xef30c340bfb3f442 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x21d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fea2f expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.418767:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x21c:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.418795:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45425a00/0xef30c340bfb3f434 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x21c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe9fe expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.418824:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x21b:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.418922:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada604f9000/0xef30c340bfb3f426 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x21b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe9cd expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.418945:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x21a:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.418961:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada604f9c00/0xef30c340bfb3f418 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x21a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe99c expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.418984:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x219:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.418996:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50b60400/0xef30c340bfb3f40a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x219:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe96b expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.419013:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x218:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.419025:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50b61600/0xef30c340bfb3f3fc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x218:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe93a expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.419041:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x217:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.419054:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45425200/0xef30c340bfb3f3ee lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x217:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe909 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.419069:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x216:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.419082:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45424400/0xef30c340bfb3f3e0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x216:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe8d8 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.419096:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x215:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.419108:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45424000/0xef30c340bfb3f3d2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x215:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe8a7 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.419122:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x214:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.419134:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45424c00/0xef30c340bfb3f3c4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x214:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe876 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.419149:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x213:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.419163:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45424200/0xef30c340bfb3f3b6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x213:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe845 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.419184:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x212:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.419199:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45424800/0xef30c340bfb3f3a8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x212:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe814 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.419213:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x211:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.419224:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50b60000/0xef30c340bfb3f39a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x211:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe7e3 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.419239:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x210:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.419249:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50b60200/0xef30c340bfb3f38c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x210:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe7b2 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.419264:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x20f:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.419276:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50b60800/0xef30c340bfb3f37e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x20f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe781 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.419291:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x20e:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.419302:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50b60a00/0xef30c340bfb3f370 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x20e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe750 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.419316:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x20d:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.419516:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50b61800/0xef30c340bfb3f362 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x20d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe71f expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.419541:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x20c:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.419628:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50b61a00/0xef30c340bfb3f354 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x20c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe6ee expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.419650:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x20b:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.419663:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50b61e00/0xef30c340bfb3f346 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x20b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe6bd expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.419678:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x20a:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.419689:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50b60e00/0xef30c340bfb3f338 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x20a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe68c expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.419702:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x209:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.419713:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50b60c00/0xef30c340bfb3f32a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x209:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe65b expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.419729:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x208:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.419742:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50b61200/0xef30c340bfb3f31c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x208:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe62a expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.419757:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x207:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.419770:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606e7a00/0xef30c340bfb3f30e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x207:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe5f9 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.419791:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x206:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.419803:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada605ffc00/0xef30c340bfb3f300 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x206:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe5c8 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.419819:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x205:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.422571:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada605ffa00/0xef30c340bfb3f2f2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x205:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe597 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.422668:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x204:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.422715:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada605fee00/0xef30c340bfb3f2e4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x204:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe566 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.422735:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x203:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.422755:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada605fe800/0xef30c340bfb3f2d6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x203:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe535 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.422771:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x202:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.422783:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada605ff400/0xef30c340bfb3f2c8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x202:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe504 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.422801:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x201:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.422818:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50b60600/0xef30c340bfb3f2ba lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x201:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe4d3 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.422837:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x200:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.422851:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50b61000/0xef30c340bfb3f2ac lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x200:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe4a2 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.422889:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1ff:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.422904:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50b61c00/0xef30c340bfb3f29e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1ff:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe471 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.422920:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1fe:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.422932:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada605fe000/0xef30c340bfb3f290 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1fe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe440 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.422953:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1fd:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.422963:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada605ff200/0xef30c340bfb3f282 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1fd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe40f expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.422976:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1fc:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.422987:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada605fe600/0xef30c340bfb3f274 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1fc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe3de expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.423001:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1fb:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.423012:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada605fec00/0xef30c340bfb3f266 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1fb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe3ad expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.423027:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1fa:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.423039:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50b83c00/0xef30c340bfb3f258 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1fa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe37c expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.423057:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1f9:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.423069:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5176e600/0xef30c340bfb3f24a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1f9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe34b expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.423083:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1f8:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.423095:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50b83e00/0xef30c340bfb3f23c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1f8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe31a expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.423110:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1f7:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.423126:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50b83200/0xef30c340bfb3f22e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1f7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe2e9 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.423148:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1f6:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.423160:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50b82600/0xef30c340bfb3f220 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1f6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe2b8 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.423175:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1f5:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.423186:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50b83800/0xef30c340bfb3f212 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1f5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe287 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.423202:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1f4:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.423215:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50b82e00/0xef30c340bfb3f204 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1f4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe256 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.423230:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1f3:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.423241:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50b82200/0xef30c340bfb3f1f6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1f3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe225 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.423260:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1f2:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.423272:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada605fe200/0xef30c340bfb3f1e8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1f2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe1f4 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.423291:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1f1:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.423306:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada605fe400/0xef30c340bfb3f1da lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1f1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe1c3 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.423322:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1f0:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.423335:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c200/0xef30c340bfb3f1cc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1f0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe192 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.423357:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1ef:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.423369:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50b82c00/0xef30c340bfb3f1be lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1ef:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe161 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.423383:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1ee:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.423396:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50b83a00/0xef30c340bfb3f1b0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1ee:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe130 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.423408:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1ed:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.423420:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50b83400/0xef30c340bfb3f1a2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1ed:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe0ff expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.423434:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1ec:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.423445:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada605ffe00/0xef30c340bfb3f194 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1ec:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe0ce expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.423461:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1eb:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.423471:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada605fea00/0xef30c340bfb3f186 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1eb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe09d expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.424106:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1ea:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.424137:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50b82a00/0xef30c340bfb3f178 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1ea:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe06c expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.424159:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1e9:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.424176:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d000/0xef30c340bfb3f16a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1e9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe03b expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.424198:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1e8:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.424217:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606b6600/0xef30c340bfb3f15c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1e8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fe00a expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.424238:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1e7:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.424250:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51763c00/0xef30c340bfb3f14e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1e7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fdfd9 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.424264:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1e6:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.424276:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada7d011200/0xef30c340bfb3f140 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1e6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fdfa8 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.424294:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1e5:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.424307:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d200/0xef30c340bfb3f132 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1e5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fdf77 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.424326:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1e4:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.424353:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c800/0xef30c340bfb3f124 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1e4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fdf46 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.424371:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1e3:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.424386:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50b82000/0xef30c340bfb3f116 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1e3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fdf15 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.424403:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1e2:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.424416:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ca00/0xef30c340bfb3f108 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1e2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fdee4 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.424436:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1e1:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.424452:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d600/0xef30c340bfb3f0fa lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1e1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fdeb3 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.424467:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1e0:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.424492:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada605ff600/0xef30c340bfb3f0ec lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1e0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fde82 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.424512:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1df:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.424528:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada605ff800/0xef30c340bfb3f0de lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1df:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fde51 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.424988:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1de:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.425013:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50b82400/0xef30c340bfb3f0d0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1de:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fde20 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.425034:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1dd:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.425047:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4542b000/0xef30c340bfb3f0c2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1dd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fddef expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.425068:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1dc:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.425081:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4542bc00/0xef30c340bfb3f0b4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1dc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fddbe expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.425100:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1db:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.425115:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4542a600/0xef30c340bfb3f0a6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1db:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fdd8d expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.425139:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1da:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.425153:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4542a200/0xef30c340bfb3f098 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1da:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fdd5c expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.425170:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1d9:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.425184:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4542a000/0xef30c340bfb3f08a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1d9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fdd2b expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.425201:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1d8:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.425213:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4542ac00/0xef30c340bfb3f07c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1d8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fdcfa expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.425231:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1d7:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.425244:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4542aa00/0xef30c340bfb3f06e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1d7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fdcc9 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.425260:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1d6:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.425271:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4542a800/0xef30c340bfb3f060 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1d6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fdc98 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.425286:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1d5:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.425297:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4542a400/0xef30c340bfb3f052 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1d5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fdc67 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.425313:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1d4:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.425326:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4542be00/0xef30c340bfb3f044 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1d4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fdc36 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.425420:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1d3:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.425437:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50b83600/0xef30c340bfb3f036 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1d3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fdc05 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.425452:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1d2:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.425463:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50b83000/0xef30c340bfb3f028 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1d2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fdbd4 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.425478:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1d1:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.425507:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51666a00/0xef30c340bfb3f01a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1d1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fdba3 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.425524:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1d0:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.425541:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c000/0xef30c340bfb3f00c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1d0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fdb72 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.425635:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1cf:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.425651:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172dc00/0xef30c340bfb3effe lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1cf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fdb41 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.425668:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1ce:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.425681:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4542b600/0xef30c340bfb3eff0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1ce:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fdb10 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.425697:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1cd:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.425709:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4542b800/0xef30c340bfb3efe2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1cd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fdadf expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.425730:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1cc:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.425744:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5176e000/0xef30c340bfb3efd4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1cc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fdaae expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.425762:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1cb:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.425775:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5176e200/0xef30c340bfb3efc6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1cb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fda7d expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.425790:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1ca:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.425802:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5176ea00/0xef30c340bfb3efb8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1ca:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fda4c expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.425981:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1c9:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.425998:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5176f200/0xef30c340bfb3efaa lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1c9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fda1b expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.426014:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1c8:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.426026:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5176e800/0xef30c340bfb3ef9c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1c8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd9ea expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.426043:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1c7:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.426055:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5176fa00/0xef30c340bfb3ef8e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1c7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd9b9 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.426071:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1c6:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.426083:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51762200/0xef30c340bfb3ef80 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1c6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd988 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.426107:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1c5:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.426120:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172ce00/0xef30c340bfb3ef72 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1c5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd957 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.426137:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1c4:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.426149:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172c400/0xef30c340bfb3ef64 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1c4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd926 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.426165:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1c3:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.426177:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172d800/0xef30c340bfb3ef56 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1c3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd8f5 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.426192:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1c2:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.426206:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5176f600/0xef30c340bfb3ef48 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1c2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd8c4 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.426222:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1c1:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.426234:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5176f800/0xef30c340bfb3ef3a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1c1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd893 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.426251:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1c0:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.426263:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5172cc00/0xef30c340bfb3ef2c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1c0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd862 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.426277:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1bf:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.426289:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5176f000/0xef30c340bfb3ef1e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1bf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd831 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.426307:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1be:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.426319:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5176e400/0xef30c340bfb3ef10 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1be:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd800 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.426717:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1bd:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.426745:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5176f400/0xef30c340bfb3ef02 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1bd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd7cf expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.426766:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1bc:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.426781:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4542ba00/0xef30c340bfb3eef4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1bc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd79e expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.426799:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1bb:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.426956:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4542ae00/0xef30c340bfb3eee6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1bb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd76d expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.426980:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1ba:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.426994:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4542b400/0xef30c340bfb3eed8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1ba:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd73c expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.427010:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1b9:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.427023:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4542b200/0xef30c340bfb3eeca lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1b9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd70b expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.427039:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1b8:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.427054:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60767800/0xef30c340bfb3eebc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1b8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd6da expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.427075:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1b7:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.427090:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606e7c00/0xef30c340bfb3eeae lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1b7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd6a9 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.427107:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1b6:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.427120:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606e7800/0xef30c340bfb3eea0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1b6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd678 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.427136:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1b5:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.427152:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5176fc00/0xef30c340bfb3ee92 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1b5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd647 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.427168:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1b4:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.427181:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5176ee00/0xef30c340bfb3ee84 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1b4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd616 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.427199:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1b3:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.427212:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5176fe00/0xef30c340bfb3ee76 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1b3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd5e5 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.427228:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1b2:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.427242:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada5176ec00/0xef30c340bfb3ee68 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1b2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd5b4 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.427257:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1b1:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.427270:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51636e00/0xef30c340bfb3ee5a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1b1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd583 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.427292:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1b0:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.427305:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51667a00/0xef30c340bfb3ee4c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1b0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd552 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.427321:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1af:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.427569:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51666600/0xef30c340bfb3ee3e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1af:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd521 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.427594:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1ae:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.427611:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51666e00/0xef30c340bfb3ee30 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1ae:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd4f0 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.427628:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1ad:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.427644:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51666c00/0xef30c340bfb3ee22 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1ad:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd4bf expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.427660:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1ac:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.427671:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51666400/0xef30c340bfb3ee14 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1ac:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd48e expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.427690:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1ab:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.427702:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51667200/0xef30c340bfb3ee06 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1ab:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd45d expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.427719:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1aa:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.427733:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51667e00/0xef30c340bfb3edf8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1aa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd42c expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.427754:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1a9:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.427767:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51667c00/0xef30c340bfb3edea lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1a9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd3fb expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.427782:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1a8:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.427793:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51666000/0xef30c340bfb3eddc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1a8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd3ca expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.427822:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1a7:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.427995:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51666200/0xef30c340bfb3edce lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1a7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd399 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.428017:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1a6:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.428031:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51667400/0xef30c340bfb3edc0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1a6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd368 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.428046:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1a5:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.428058:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51667800/0xef30c340bfb3edb2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1a5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd337 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.428073:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1a4:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.428086:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606e6e00/0xef30c340bfb3eda4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1a4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd306 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.428104:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1a3:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.428119:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606e7200/0xef30c340bfb3ed96 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1a3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd2d5 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.428141:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1a2:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.428154:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606e7400/0xef30c340bfb3ed88 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1a2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd2a4 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.428169:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1a1:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.428180:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606e6000/0xef30c340bfb3ed7a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1a1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd273 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.428195:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x1a0:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.428206:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606e6c00/0xef30c340bfb3ed6c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x1a0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd242 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.428222:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x19f:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.428236:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51666800/0xef30c340bfb3ed5e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x19f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd211 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.428251:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x19e:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.428263:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606e6600/0xef30c340bfb3ed50 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x19e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd1e0 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.428279:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x19d:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.428291:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606e6200/0xef30c340bfb3ed42 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x19d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd1af expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.428306:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x19c:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.428319:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606e6a00/0xef30c340bfb3ed34 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x19c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd17e expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.428644:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x19b:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.428662:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606e6400/0xef30c340bfb3ed26 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x19b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd14d expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.428678:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x19a:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.428689:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606e6800/0xef30c340bfb3ed18 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x19a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd11c expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.428702:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x199:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.428714:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51667000/0xef30c340bfb3ed0a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x199:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd0eb expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.428728:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x198:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.428741:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51667600/0xef30c340bfb3ecfc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x198:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd0ba expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.428757:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x197:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.428768:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50b02600/0xef30c340bfb3ecee lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x197:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd089 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.428782:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x196:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.428793:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50b03a00/0xef30c340bfb3ece0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x196:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd058 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.428826:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x195:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.428960:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50b03200/0xef30c340bfb3ecd2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x195:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fd027 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.428983:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x194:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.428996:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50b03800/0xef30c340bfb3ecc4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x194:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fcff6 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.429010:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x193:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.429024:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606e7e00/0xef30c340bfb3ecb6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x193:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fcfc5 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.429040:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x192:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.429052:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606e7600/0xef30c340bfb3eca8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x192:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fcf94 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572090.429067:0:84245:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x191:0x0]: losing layout lock 00010000:00010000:1.0:1788572090.429080:0:84245:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606e7000/0xef30c340bfb3ec9a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x191:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b82fcf63 expref: -99 pid: 83612 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1788572090.429094:0:84245:0:(ldlm_request.c:2901:ldlm_cancel_unused_locks_for_replay()) Canceled 0 unused locks from namespace lustre-MDT0000-mdc-ffff9ada58b0a000 00000100:00080000:1.0:1788572090.429684:0:84245:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from REPLAY_LOCKS to REPLAY_WAIT 00000100:00100000:2.0:1788572090.430279:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449614312448:192.168.202.113@tcp:400:ldlm_lock_repla.0 00000100:00100000:1.0:1788572090.443746:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 210453399506 gen 9 00000100:00080000:1.0:1788572090.443788:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from REPLAY_WAIT to RECOVER 00000100:00080000:1.0:1788572090.443805:0:2382:0:(client.c:3248:ptlrpc_resend_req()) @@@ going to resend req@ffff9ada455e2d80 x1875449614312448/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572106 ref 1 fl Interpret:RQU/2c0/0 rc 0/0 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572090.443841:0:2382:0:(client.c:3257:ptlrpc_resend_req()) @@@ it has reply, so skip it req@ffff9ada455e2d80 x1875449614312448/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572106 ref 1 fl Interpret:RQU/2c0/0 rc 0/0 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572090.443849:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from RECOVER to FULL 00000100:02000000:1.0:1788572090.444642:0:2382:0:(import.c:1787:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff9ada58b0a000: Connection restored to 192.168.202.113@tcp (at 192.168.202.113@tcp) 00000100:00100000:1.0:1788572090.444651:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449614312448:192.168.202.113@tcp:400:ldlm_lock_repla.0 00000100:00100000:0.0:1788572090.743933:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449614312704:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572090.744047:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449614312960:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572090.744096:0:2383:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1788572090.744103:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449614312576:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572090.744137:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449614312832:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572090.767109:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449614312576:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572090.767136:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449614312832:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572090.767147:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449614312704:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572090.767158:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449614312960:192.168.202.113@tcp:400:kworker.0 00010000:00010000:1.0:1788572091.255309:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572092.279074:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572093.305004:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:1.0:1788572094.209544:0:84290:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1788572094.209565:0:84290:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:1.0:1788572094.209597:0:84290:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1788572094.209676:0:84290:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607bca00/0xef30c340bfb40d94 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 84290 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572094.209688:0:84290:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607bca00/0xef30c340bfb40d94 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 84290 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572094.209700:0:84290:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607bca00/0xef30c340bfb40d94 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 84290 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572094.209730:0:84290:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e2d80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:84290:1875449614313088:192.168.202.113@tcp:101:lfs.0 00000100:00100000:1.0:1788572094.210074:0:84290:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada465f4f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788572094.213702:0:84290:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e2d80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:84290:1875449614313088:192.168.202.113@tcp:101:lfs.0 00010000:00010000:1.0:1788572094.213737:0:84290:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607bca00/0xef30c340bfb40d94 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306c0a expref: -99 pid: 84290 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572094.213752:0:84290:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607bca00/0xef30c340bfb40d94 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306c0a expref: -99 pid: 84290 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572094.213766:0:84290:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607bca00/0xef30c340bfb40d94 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8306c0a expref: -99 pid: 84290 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572094.213783:0:84290:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607bca00/0xef30c340bfb40d94 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306c0a expref: -99 pid: 84290 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572094.213798:0:84290:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607bca00/0xef30c340bfb40d94 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8306c0a expref: -99 pid: 84290 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572094.214157:0:84290:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607bca00/0xef30c340bfb40d94 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8306c0a expref: -99 pid: 84290 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572094.214167:0:84290:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607bca00/0xef30c340bfb40d94 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8306c0a expref: -99 pid: 84290 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1788572094.214182:0:84290:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada455e2d80 x1875449614313088/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788572110 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1788572094.214198:0:84290:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607bca00/0xef30c340bfb40d94 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306c0a expref: -99 pid: 84290 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572094.214214:0:84290:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1788572094.214222:0:84290:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:2.0:1788572094.214337:0:84290:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 65536 00000080:00010000:2.0:1788572094.214346:0:84290:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb40d94 00000080:00010000:2.0:1788572094.214353:0:84290:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788572094.214356:0:84290:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb40d94 from it ffffab0ec0d17cb8 00010000:00010000:2.0:1788572094.214360:0:84290:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607bca00/0xef30c340bfb40d94 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8306c0a expref: -99 pid: 84290 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572094.214370:0:84290:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607bca00/0xef30c340bfb40d94 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306c0a expref: -99 pid: 84290 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:2.0:1788572094.215454:0:84290:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 0, flags 0, foreign 0 00000080:00200000:2.0:1788572094.215463:0:84290:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788572094.215487:0:84290:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788572094.215502:0:84290:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607bca00/0xef30c340bfb40d94 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306c0a expref: -99 pid: 84290 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788572094.215517:0:84290:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607bca00/0xef30c340bfb40d94 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306c0a expref: -99 pid: 84290 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788572094.215531:0:84290:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788572094.215537:0:84290:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788572094.215541:0:84290:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb40d94 00000080:00010000:2.0:1788572094.215547:0:84290:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb40d94 from it ffffab0ec0d17cb8 00010000:00010000:2.0:1788572094.215551:0:84290:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607bca00/0xef30c340bfb40d94 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8306c0a expref: -99 pid: 84290 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788572094.215560:0:84290:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607bca00/0xef30c340bfb40d94 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306c0a expref: -99 pid: 84290 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788572094.215690:0:84290:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788572094.215698:0:84290:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788572094.215704:0:84290:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607bca00/0xef30c340bfb40d94 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306c0a expref: -99 pid: 84290 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788572094.215712:0:84290:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607bca00/0xef30c340bfb40d94 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306c0a expref: -99 pid: 84290 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1788572094.215719:0:84290:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788572094.215723:0:84290:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788572094.215726:0:84290:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb40d94 00000080:00010000:2.0:1788572094.215730:0:84290:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb40d94 from it ffffab0ec0d17bb0 00010000:00010000:2.0:1788572094.215733:0:84290:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607bca00/0xef30c340bfb40d94 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8306c0a expref: -99 pid: 84290 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788572094.215742:0:84290:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607bca00/0xef30c340bfb40d94 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306c0a expref: -99 pid: 84290 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:2.0:1788572094.215751:0:84290:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:2.0:1788572094.215759:0:84290:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200080:2.0:1788572094.215892:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:2.0:1788572094.215911:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454f3000 uarg=0000000000000000 00800000:00000004:2.0:1788572094.215919:0:84290:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: age 3961, max_age 4010 00000100:00100000:2.0:1788572094.217524:0:84290:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:84290:1875449614313216:192.168.202.113@tcp:41:lfs.0 00000100:00100000:2.0:1788572094.217709:0:84290:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada605c26c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572094.220420:0:84290:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:84290:1875449614313216:192.168.202.113@tcp:41:lfs.0 00800000:00000004:0.0:1788572094.220456:0:84290:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: update ffff9ada4481d198 cache blocks 317326/353529 objects 1023288/1024000 00000080:00200080:0.0:1788572094.220560:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.220588:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00800000:00000004:0.0:1788572094.220597:0:84290:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: age 3961, max_age 4010 00000100:00100000:0.0:1788572094.220636:0:84290:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:84290:1875449614313344:192.168.202.113@tcp:41:lfs.0 00000100:00100000:0.0:1788572094.221412:0:84290:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada4352fb40 going to sleep for 11 seconds 00000100:00100000:0.0:1788572094.224236:0:84290:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:84290:1875449614313344:192.168.202.113@tcp:41:lfs.0 00800000:00000004:0.0:1788572094.224269:0:84290:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: update ffff9ada44819e18 cache blocks 317392/353529 objects 1023736/1024000 00000080:00200080:0.0:1788572094.224396:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224416:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224423:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224427:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224431:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224434:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224438:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224442:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224445:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224448:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224451:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224455:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224458:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224461:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224465:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224468:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224471:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224474:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224478:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224481:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224484:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224488:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224492:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224495:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224499:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224502:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224505:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224508:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224512:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224515:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224518:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224521:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224525:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224528:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224531:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224534:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224537:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224541:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224544:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224547:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224551:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224554:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224557:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224560:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224564:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224567:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224571:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224574:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224587:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224591:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224594:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224597:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224601:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224604:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224608:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224611:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224615:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224618:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224622:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224625:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224629:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224632:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224635:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224638:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224642:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224646:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224649:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224653:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224656:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224659:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224663:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224666:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224670:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224673:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224676:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224679:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224682:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224686:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224690:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224693:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224697:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224700:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224704:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224707:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224711:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224714:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224717:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224720:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224724:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224727:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224730:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224734:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224737:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224740:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224743:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224747:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224751:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224754:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224757:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224760:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224764:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224767:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224770:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224773:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224776:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224779:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224783:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224786:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224789:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224792:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224796:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224799:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224802:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224805:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224808:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224811:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224815:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224818:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224821:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224824:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224827:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224831:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224834:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224837:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224840:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00800000:00000080:0.0:1788572094.224843:0:84290:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.224847:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.224878:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00020000:00000004:0.0:1788572094.224887:0:84290:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: age 3961, max_age 4010 00000100:00100000:0.0:1788572094.224930:0:84290:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:84290:1875449614313472:192.168.202.113@tcp:13:lfs.0 00000100:00100000:0.0:1788572094.225070:0:84290:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada4352fb40 going to sleep for 11 seconds 00000100:00100000:0.0:1788572094.227389:0:84290:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:84290:1875449614313472:192.168.202.113@tcp:13:lfs.0 00020000:00000004:0.0:1788572094.227429:0:84290:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: update ffff9ada4354d198 cache blocks 901357/958279 objects 261529/262144 00000080:00200080:0.0:1788572094.228668:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.228695:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00020000:00000004:0.0:1788572094.228704:0:84290:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: age 3961, max_age 4010 00000100:00100000:0.0:1788572094.228756:0:84290:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:84290:1875449614313600:192.168.202.113@tcp:13:lfs.0 00000100:00100000:0.0:1788572094.228922:0:84290:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada4352fb40 going to sleep for 11 seconds 00000100:00100000:0.0:1788572094.231573:0:84290:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:84290:1875449614313600:192.168.202.113@tcp:13:lfs.0 00020000:00000004:0.0:1788572094.231670:0:84290:0:(obd_class.h:1116:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: update ffff9ada43549e18 cache blocks 901363/958279 objects 261534/262144 00000080:00200080:0.0:1788572094.231748:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.231767:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.231773:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.231777:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.231781:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.231785:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.231789:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.231793:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.231796:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.231799:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.231803:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.231806:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.231809:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.231812:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.231815:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.231817:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.231820:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.231823:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.231826:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.231829:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.231833:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.231836:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.231839:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.231842:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232382:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232398:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232407:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232411:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232414:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232418:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232422:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232426:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232430:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232433:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232437:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232441:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232445:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232448:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232453:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232456:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232460:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232463:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232467:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232470:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232474:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232477:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232481:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232484:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232488:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232491:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232495:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232498:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232502:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232505:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232509:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232513:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232517:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232520:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232523:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232527:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232530:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232534:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232537:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232541:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232544:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232547:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232551:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232554:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232558:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232561:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232564:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232568:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232571:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232574:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232592:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232596:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232600:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232603:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232606:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232610:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232614:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232617:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232621:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232624:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232628:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232631:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232635:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232638:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232642:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232645:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232649:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232652:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232656:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232659:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232663:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232666:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232670:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232673:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232676:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232680:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232683:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232686:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232690:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232693:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232697:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232701:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232704:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232708:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232711:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232715:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232718:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232722:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232726:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232729:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232732:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232735:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232739:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232743:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232747:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232750:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232753:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232756:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232760:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232763:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200080:0.0:1788572094.232766:0:84290:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdbb2fc720 00020000:00000080:0.0:1788572094.232770:0:84290:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5060b400 uarg=0000000000000000 00000080:00200008:0.0:1788572094.232788:0:84290:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00010000:00010000:0.0:1788572094.327986:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572095.354172:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572095.864044:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449614313728:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572095.864181:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614313984:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572095.864223:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614314240:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572095.864253:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1788572095.864259:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614313856:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572095.864298:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614314112:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572095.876570:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614313856:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572095.876592:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614314112:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572095.876604:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614313984:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572095.876626:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614314240:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572095.879042:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449614313728:192.168.202.113@tcp:400:kworker.0 00010000:00010000:1.0:1788572096.381000:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572097.399221:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572098.424143:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572099.447980:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572100.474849:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572100.983905:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449614314368:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572100.984002:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449614314624:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572100.984032:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449614314880:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572100.984078:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1788572100.984085:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449614314496:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572100.984108:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449614314752:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572100.992140:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449614314752:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572100.992158:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449614314624:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572100.992168:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449614314880:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572100.995918:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449614314496:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572100.995937:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449614314368:192.168.202.113@tcp:400:kworker.0 00010000:00010000:1.0:1788572101.502391:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:0.0:1788572101.615763:0:84572:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 00010000:00010000:0.0:1788572102.519324:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:0.0:1788572103.332230:0:84720:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec 00010000:00010000:1.0:1788572103.547776:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572104.568246:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:3.0:1788572105.583246:0:84758:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1788572105.583263:0:84758:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:3.0:1788572105.583288:0:84758:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1788572105.583309:0:84758:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607bca00/0xef30c340bfb40d94 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306c0a expref: -99 pid: 84290 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1788572105.583328:0:84758:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607bca00/0xef30c340bfb40d94 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306c0a expref: -99 pid: 84290 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:3.0:1788572105.583337:0:84758:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1788572105.583345:0:84758:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:3.0:1788572105.583348:0:84758:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb40d94 00000080:00010000:3.0:1788572105.583357:0:84758:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb40d94 from it ffffab0ec1c17cb8 00010000:00010000:3.0:1788572105.583362:0:84758:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607bca00/0xef30c340bfb40d94 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8306c0a expref: -99 pid: 84290 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1788572105.583372:0:84758:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607bca00/0xef30c340bfb40d94 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306c0a expref: -99 pid: 84290 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00200008:3.0:1788572105.583975:0:84758:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 0, flags 0, foreign 0 00000080:00200000:3.0:1788572105.583981:0:84758:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:3.0:1788572105.583993:0:84758:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1788572105.584001:0:84758:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607bca00/0xef30c340bfb40d94 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306c0a expref: -99 pid: 84290 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:3.0:1788572105.584012:0:84758:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607bca00/0xef30c340bfb40d94 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306c0a expref: -99 pid: 84290 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00200000:3.0:1788572105.584018:0:84758:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1788572105.584022:0:84758:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:3.0:1788572105.584025:0:84758:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb40d94 00000080:00010000:3.0:1788572105.584029:0:84758:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb40d94 from it ffffab0ec1c17cb8 00010000:00010000:3.0:1788572105.584032:0:84758:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607bca00/0xef30c340bfb40d94 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8306c0a expref: -99 pid: 84290 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:3.0:1788572105.584039:0:84758:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607bca00/0xef30c340bfb40d94 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306c0a expref: -99 pid: 84290 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:3.0:1788572105.584136:0:84758:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:3.0:1788572105.584145:0:84758:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1788572105.584152:0:84758:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607bca00/0xef30c340bfb40d94 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306c0a expref: -99 pid: 84290 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:3.0:1788572105.584160:0:84758:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607bca00/0xef30c340bfb40d94 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306c0a expref: -99 pid: 84290 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:3.0:1788572105.584166:0:84758:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1788572105.584171:0:84758:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:3.0:1788572105.584173:0:84758:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb40d94 00000080:00010000:3.0:1788572105.584177:0:84758:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb40d94 from it ffffab0ec1c17bb0 00010000:00010000:3.0:1788572105.584181:0:84758:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607bca00/0xef30c340bfb40d94 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8306c0a expref: -99 pid: 84290 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:3.0:1788572105.584188:0:84758:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607bca00/0xef30c340bfb40d94 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8306c0a expref: -99 pid: 84290 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000080:00200000:3.0:1788572105.584196:0:84758:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:3.0:1788572105.584203:0:84758:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200080:3.0:1788572105.588911:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.588951:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00800000:00000004:3.0:1788572105.588963:0:84758:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: age 4011, max_age 4022 00000100:00100000:3.0:1788572105.589013:0:84758:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c1c00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:84758:1875449614315008:192.168.202.113@tcp:41:lfs.0 00000100:00100000:3.0:1788572105.589111:0:84758:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6071a900 going to sleep for 11 seconds 00010000:00010000:3.0:1788572105.592419:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572105.594076:0:84758:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c1c00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:84758:1875449614315008:192.168.202.113@tcp:41:lfs.0 00800000:00000004:3.0:1788572105.594106:0:84758:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: update ffff9ada4481d198 cache blocks 317326/353529 objects 1023288/1024000 00000080:00200080:3.0:1788572105.594234:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.594252:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00800000:00000004:3.0:1788572105.594260:0:84758:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: age 4011, max_age 4022 00000100:00100000:3.0:1788572105.594295:0:84758:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c1c00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:84758:1875449614315136:192.168.202.113@tcp:41:lfs.0 00000100:00100000:3.0:1788572105.594375:0:84758:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6071a900 going to sleep for 11 seconds 00000100:00100000:3.0:1788572105.599676:0:84758:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c1c00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:84758:1875449614315136:192.168.202.113@tcp:41:lfs.0 00800000:00000004:3.0:1788572105.599708:0:84758:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: update ffff9ada44819e18 cache blocks 317392/353529 objects 1023736/1024000 00000080:00200080:3.0:1788572105.599786:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.599806:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.599814:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.599818:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.599822:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.599826:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.599829:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.599832:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.599835:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.599838:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.599841:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.599846:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600145:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600156:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600164:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600168:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600172:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600176:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600180:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600183:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600187:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600190:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600193:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600197:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600201:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600204:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600208:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600213:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600216:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600219:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600223:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600227:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600232:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600236:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600240:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600244:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600249:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600253:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600257:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600261:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600265:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600268:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600273:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600276:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600280:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600283:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600287:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600291:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600296:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600300:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600304:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600308:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600312:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600316:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600320:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600324:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600328:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600332:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600336:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600340:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600344:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600349:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600353:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600357:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600361:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600365:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600369:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600373:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600378:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600382:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600386:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600390:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600394:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600398:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600402:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600405:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600409:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600413:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600417:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600420:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600424:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600428:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600432:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600436:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600440:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600444:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600449:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600452:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600456:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600460:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600464:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600468:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600472:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600475:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.600479:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.600483:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.601738:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.601750:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.601755:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.601758:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.601762:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.601765:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.601769:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.601773:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.601777:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.601780:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.601784:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.601787:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.601791:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.601795:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.601799:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.601804:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.601808:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.601812:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.601816:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.601819:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.601823:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.601826:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.601829:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.601833:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.601837:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.601840:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.601844:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.601848:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.601925:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00800000:00000080:3.0:1788572105.601933:0:84758:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.601950:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.601957:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00020000:00000004:3.0:1788572105.601967:0:84758:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: age 4011, max_age 4022 00000100:00100000:3.0:1788572105.602015:0:84758:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c1c00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:84758:1875449614315264:192.168.202.113@tcp:13:lfs.0 00000100:00100000:3.0:1788572105.602157:0:84758:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6071a900 going to sleep for 11 seconds 00000100:00100000:3.0:1788572105.606098:0:84758:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c1c00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:84758:1875449614315264:192.168.202.113@tcp:13:lfs.0 00020000:00000004:3.0:1788572105.606133:0:84758:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: update ffff9ada4354d198 cache blocks 901357/958279 objects 261529/262144 00000080:00200080:3.0:1788572105.606257:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.606275:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00020000:00000004:3.0:1788572105.606286:0:84758:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: age 4011, max_age 4022 00000100:00100000:3.0:1788572105.606322:0:84758:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c1c00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:84758:1875449614315392:192.168.202.113@tcp:13:lfs.0 00000100:00100000:3.0:1788572105.606396:0:84758:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6071a900 going to sleep for 11 seconds 00000100:00100000:3.0:1788572105.614400:0:84758:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c1c00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:84758:1875449614315392:192.168.202.113@tcp:13:lfs.0 00020000:00000004:3.0:1788572105.614432:0:84758:0:(obd_class.h:1116:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: update ffff9ada43549e18 cache blocks 901363/958279 objects 261534/262144 00000080:00200080:3.0:1788572105.614679:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.614700:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.614708:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.614713:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.614718:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.614722:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.614726:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.614730:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.614734:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.614738:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.614742:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.614746:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.614750:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.614753:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.614757:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.614761:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.614764:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.614767:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.614770:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.614774:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.614778:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.614782:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.614786:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.614789:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.614793:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.614797:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.614800:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.614804:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.614808:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.614811:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.614816:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.614819:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.614823:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.614827:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.614831:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.614835:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.614838:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.614842:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.614846:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.614849:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.614882:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.614888:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.614892:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.614896:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.614900:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.614904:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.614908:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.614911:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.614915:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.614919:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.614922:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.614925:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.614928:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.614931:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.614947:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.614951:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.614955:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.614958:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.614962:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.614965:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.614969:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.614973:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.614977:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.614980:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.614984:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.614988:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.614992:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.614996:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.615000:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.615005:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.615009:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.615013:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.615017:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.615021:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.615025:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.615029:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.615033:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.615036:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.615040:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.615043:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.615047:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.615051:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.615054:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.615058:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.615062:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.615066:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.615070:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.615073:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.615078:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.615081:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.615085:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.615089:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.615093:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.615096:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.615101:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.615105:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.615108:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.615112:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.615117:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.615120:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.615124:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.615128:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.615131:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.615135:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.615139:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.615143:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.615147:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.615150:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.615155:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.615159:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.615163:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.615167:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.615171:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.615175:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.615179:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.615182:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.615187:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.615190:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.615194:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.615198:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.615201:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.615204:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.615208:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.615212:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200080:3.0:1788572105.615216:0:84758:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe5c4e47b0 00020000:00000080:3.0:1788572105.615220:0:84758:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5165f800 uarg=0000000000000000 00000080:00200008:3.0:1788572105.615234:0:84758:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000100:00100000:2.0:1788572106.103891:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449614315520:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572106.118262:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449614315520:192.168.202.113@tcp:400:kworker.0 00010000:00010000:1.0:1788572106.615535:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572107.641690:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788572116.343596:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449614520832:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572116.346144:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449614521088:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572116.346210:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449614521344:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572116.350501:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1788572116.350533:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449614520960:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572116.350586:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449614521216:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572116.369574:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449614520960:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572116.369592:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449614521216:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572116.369602:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449614520832:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572116.369614:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449614521088:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572116.369626:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449614521344:192.168.202.113@tcp:400:kworker.0 00010000:00010000:1.0:1788572116.855591:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572117.884776:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572118.903287:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:0.0:1788572119.771407:0:83353:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572119.771447:0:83353:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572119.771543:0:83353:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51636000/0xef30c340bfb42374 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 83353 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572119.771555:0:83353:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51636000/0xef30c340bfb42374 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 83353 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572119.771570:0:83353:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51636000/0xef30c340bfb42374 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 83353 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572119.771602:0:83353:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37800 pname:cluuid:pid:xid:nid:opc:job bash:35eb2d60-2de0-4b00-9815-d5153edd0057:83353:1875449614521472:192.168.202.113@tcp:101:bash.0 00000100:00100000:0.0:1788572119.771726:0:83353:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada606dea80 going to sleep for 11 seconds 00000100:00100000:0.0:1788572119.781005:0:83353:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748365200 gen 9 00000100:00100000:0.0:1788572119.785993:0:83353:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37800 pname:cluuid:pid:xid:nid:opc:job bash:35eb2d60-2de0-4b00-9815-d5153edd0057:83353:1875449614521472:192.168.202.113@tcp:101:bash.0 00010000:00010000:0.0:1788572119.786395:0:83353:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51636000/0xef30c340bfb42374 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b830e429 expref: -99 pid: 83353 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572119.786411:0:83353:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51636000/0xef30c340bfb42374 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b830e429 expref: -99 pid: 83353 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572119.786424:0:83353:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51636000/0xef30c340bfb42374 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b830e429 expref: -99 pid: 83353 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572119.786437:0:83353:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51636000/0xef30c340bfb42374 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b830e429 expref: -99 pid: 83353 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572119.786452:0:83353:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51636000/0xef30c340bfb42374 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b830e429 expref: -99 pid: 83353 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572119.786461:0:83353:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51636000/0xef30c340bfb42374 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b830e429 expref: -99 pid: 83353 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572119.786468:0:83353:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51636000/0xef30c340bfb42374 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b830e429 expref: -99 pid: 83353 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788572119.786487:0:83353:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada60a37800 x1875449614521472/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788572135 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788572119.786502:0:83353:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51636000/0xef30c340bfb42374 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b830e429 expref: -99 pid: 83353 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572119.786519:0:83353:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1788572119.786528:0:83353:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1788572119.786578:0:83353:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 65536 00000080:00010000:0.0:1788572119.786585:0:83353:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb42374 00000080:00010000:0.0:1788572119.786592:0:83353:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572119.786595:0:83353:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb42374 from it ffffab0ec1c3fbb0 00010000:00010000:0.0:1788572119.786599:0:83353:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51636000/0xef30c340bfb42374 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b830e429 expref: -99 pid: 83353 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572119.786609:0:83353:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51636000/0xef30c340bfb42374 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b830e429 expref: -99 pid: 83353 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572119.789393:0:83353:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:0.0:1788572119.789423:0:83353:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 304000 00000080:00200008:0.0:1788572119.789467:0:83353:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000002:00010000:0.0:1788572119.789505:0:83353:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: readdir flags 0 00010000:00010000:0.0:1788572119.789528:0:83353:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51636000/0xef30c340bfb42374 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b830e429 expref: -99 pid: 83353 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788572119.789556:0:83353:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51636000/0xef30c340bfb42374 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b830e429 expref: -99 pid: 83353 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000100:00100000:0.0:1788572119.793926:0:83353:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37800 pname:cluuid:pid:xid:nid:opc:job bash:35eb2d60-2de0-4b00-9815-d5153edd0057:83353:1875449614521600:192.168.202.113@tcp:37:bash.0 00000100:00100000:0.0:1788572119.794075:0:83353:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada606dea80 going to sleep for 11 seconds 00000100:00100000:0.0:1788572119.818925:0:83353:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37800 pname:cluuid:pid:xid:nid:opc:job bash:35eb2d60-2de0-4b00-9815-d5153edd0057:83353:1875449614521600:192.168.202.113@tcp:37:bash.0 00010000:00010000:0.0:1788572119.819699:0:83353:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51636000/0xef30c340bfb42374 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b830e429 expref: -99 pid: 83353 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788572119.819720:0:83353:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51636000/0xef30c340bfb42374 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b830e429 expref: -99 pid: 83353 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1788572119.820717:0:83353:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:1.0:1788572119.820758:0:83353:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1788572119.820780:0:83353:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51636000/0xef30c340bfb42374 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b830e429 expref: -99 pid: 83353 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1788572119.820798:0:83353:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51636000/0xef30c340bfb42374 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b830e429 expref: -99 pid: 83353 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1788572119.820807:0:83353:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1788572119.820813:0:83353:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:1.0:1788572119.820817:0:83353:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb42374 00000080:00010000:1.0:1788572119.820825:0:83353:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb42374 from it ffffab0ec1c3fcd8 00010000:00010000:1.0:1788572119.820828:0:83353:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51636000/0xef30c340bfb42374 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b830e429 expref: -99 pid: 83353 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1788572119.820836:0:83353:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51636000/0xef30c340bfb42374 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b830e429 expref: -99 pid: 83353 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00000080:00200000:1.0:1788572119.822820:0:83353:0:(dir.c:332:ll_iterate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) pos/size0/65536 32bit_api 0 00000080:00200000:1.0:1788572119.823013:0:83353:0:(dir.c:332:ll_iterate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) pos/size18446744073709551614/65536 32bit_api 0 00000080:00200008:1.0:1788572119.823473:0:83353:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 304000 00000080:00200000:3.0:1788572119.848084:0:84976:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:3.0:1788572119.848128:0:84976:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1788572119.848152:0:84976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51636000/0xef30c340bfb42374 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b830e429 expref: -99 pid: 83353 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00010000:00010000:3.0:1788572119.848171:0:84976:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51636000/0xef30c340bfb42374 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b830e429 expref: -99 pid: 83353 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00000080:00200000:3.0:1788572119.848180:0:84976:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1788572119.848189:0:84976:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:3.0:1788572119.848193:0:84976:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb42374 00000080:00010000:3.0:1788572119.848203:0:84976:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb42374 from it ffffab0ec1c97b58 00010000:00010000:3.0:1788572119.848207:0:84976:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51636000/0xef30c340bfb42374 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b830e429 expref: -99 pid: 83353 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00010000:00010000:3.0:1788572119.848218:0:84976:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51636000/0xef30c340bfb42374 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b830e429 expref: -99 pid: 83353 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00200000:3.0:1788572119.848241:0:84976:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:3.0:1788572119.848260:0:84976:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f38.replay-single-*, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), flags=8 00000080:00200000:3.0:1788572119.848265:0:84976:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f38.replay-single-*, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=getattr 00000002:00010000:3.0:1788572119.848280:0:84976:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f38.replay-single-*,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 01 00010000:00010000:3.0:1788572119.848338:0:84976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45518200/0xef30c340bfb4237b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 84976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572119.848347:0:84976:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45518200/0xef30c340bfb4237b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 84976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572119.848358:0:84976:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45518200/0xef30c340bfb4237b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 84976 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572119.848389:0:84976:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c0380 pname:cluuid:pid:xid:nid:opc:job checkstat:35eb2d60-2de0-4b00-9815-d5153edd0057:84976:1875449614521728:192.168.202.113@tcp:101:checkstat.0 00000100:00100000:3.0:1788572119.848507:0:84976:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6071a300 going to sleep for 11 seconds 00000100:00100000:1.0:1788572119.854494:0:84976:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c0380 pname:cluuid:pid:xid:nid:opc:job checkstat:35eb2d60-2de0-4b00-9815-d5153edd0057:84976:1875449614521728:192.168.202.113@tcp:101:checkstat.0 00010000:00010000:1.0:1788572119.854527:0:84976:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45518200/0xef30c340bfb4237b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 84976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572119.854559:0:84976:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45518200/0xef30c340bfb4237b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 84976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572119.854571:0:84976:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45518200/0xef30c340bfb4237b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 84976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572119.854579:0:84976:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45518200/0xef30c340bfb4237b lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 84976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572119.854588:0:84976:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45518200/0xef30c340bfb4237b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 84976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572119.854594:0:84976:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada45518200 already unused, calling callback (ffffffffc1966630) 00010000:00010000:1.0:1788572119.854599:0:84976:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45518200/0xef30c340bfb4237b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 84976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572119.854607:0:84976:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45518200/0xef30c340bfb4237b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 84976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572119.854615:0:84976:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45518200/0xef30c340bfb4237b lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 84976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572119.854623:0:84976:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1788572119.854631:0:84976:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45518200/0xef30c340bfb4237b lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 84976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572119.854648:0:84976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45518200/0xef30c340bfb4237b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 84976 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1788572119.854668:0:84976:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9ada454c0380 x1875449614521728/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/584 e 0 to 0 dl 1788572135 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'checkstat.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1788572119.854684:0:84976:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:1.0:1788572119.854718:0:84976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51636000/0xef30c340bfb42374 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b830e429 expref: -99 pid: 83353 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:1.0:1788572119.854732:0:84976:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51636000/0xef30c340bfb42374 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b830e429 expref: -99 pid: 83353 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00010000:1.0:1788572119.854742:0:84976:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb42374 from it ffffab0ec1c97930 00010000:00010000:1.0:1788572119.854746:0:84976:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51636000/0xef30c340bfb42374 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b830e429 expref: -99 pid: 83353 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:1.0:1788572119.854755:0:84976:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51636000/0xef30c340bfb42374 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b830e429 expref: -99 pid: 83353 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:1.0:1788572119.928280:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572120.954552:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572121.474396:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614521984:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572121.474530:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a36680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614522240:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572121.474568:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:0.0:1788572121.474586:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449614521856:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572121.474618:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614522112:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572121.474641:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614522368:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572121.481065:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449614521856:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572121.481094:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a35c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614522112:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572121.481119:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614522368:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572121.481132:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a35880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614521984:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572121.481147:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a36680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614522240:192.168.202.113@tcp:400:kworker.0 00010000:00010000:1.0:1788572121.975006:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572122.999652:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572124.036950:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572125.048269:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572126.082633:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572126.584362:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449614522496:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572126.584464:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614522752:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572126.584490:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614523008:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572126.584516:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1788572126.584522:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614522624:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572126.584543:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614522880:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572126.604115:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614522624:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572126.604136:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449614522496:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572126.604149:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614522752:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572126.604163:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614523008:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572126.618455:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449614522880:192.168.202.113@tcp:400:kworker.0 00010000:00010000:3.0:1788572127.099732:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572128.118984:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1788572128.477608:0:85541:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == replay-single test 39: test recovery from unlink llog (test llog_gen_rec) ========================================================== 21:35:26 (1788572126) 00010000:00010000:2.0:1788572129.143582:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572130.169630:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572149.626393:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572150.648271:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572151.672669:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572152.183209:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a36300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615037184:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572152.183314:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615037440:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572152.183349:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:0.0:1788572152.183355:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449615037056:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572152.183387:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615037312:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572152.183413:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a34700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615037568:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572152.191595:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a35500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615037312:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572152.191631:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a36300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615037184:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572152.193992:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a34700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615037568:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572152.194009:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615037440:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572152.205078:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449615037056:192.168.202.113@tcp:400:kworker.0 00000080:00200008:0.0:1788572152.382956:0:85754:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1788572152.382971:0:85754:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572152.382998:0:85754:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572152.383066:0:85754:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779c00/0xef30c340bfb44f3b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 85754 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572152.383079:0:85754:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779c00/0xef30c340bfb44f3b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 85754 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572152.383090:0:85754:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779c00/0xef30c340bfb44f3b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 85754 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572152.383119:0:85754:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37480 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:85754:1875449615037696:192.168.202.113@tcp:101:lfs.0 00000100:00100000:0.0:1788572152.383211:0:85754:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada455ea9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572152.387548:0:85754:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572152.389453:0:85754:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37480 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:85754:1875449615037696:192.168.202.113@tcp:101:lfs.0 00010000:00010000:0.0:1788572152.389484:0:85754:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779c00/0xef30c340bfb44f3b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8317d8f expref: -99 pid: 85754 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572152.389498:0:85754:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779c00/0xef30c340bfb44f3b lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8317d8f expref: -99 pid: 85754 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572152.389512:0:85754:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779c00/0xef30c340bfb44f3b lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8317d8f expref: -99 pid: 85754 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572152.389525:0:85754:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779c00/0xef30c340bfb44f3b lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8317d8f expref: -99 pid: 85754 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572152.389540:0:85754:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779c00/0xef30c340bfb44f3b lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8317d8f expref: -99 pid: 85754 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572152.389549:0:85754:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779c00/0xef30c340bfb44f3b lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8317d8f expref: -99 pid: 85754 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572152.389565:0:85754:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779c00/0xef30c340bfb44f3b lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8317d8f expref: -99 pid: 85754 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788572152.389578:0:85754:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada60a37480 x1875449615037696/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788572168 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788572152.389593:0:85754:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779c00/0xef30c340bfb44f3b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8317d8f expref: -99 pid: 85754 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572152.389608:0:85754:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1788572152.390399:0:85754:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1788572152.390434:0:85754:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:0.0:1788572152.390441:0:85754:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb44f3b 00000080:00010000:0.0:1788572152.390447:0:85754:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572152.390451:0:85754:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb44f3b from it ffffab0ec1c57cb8 00010000:00010000:0.0:1788572152.390455:0:85754:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779c00/0xef30c340bfb44f3b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8317d8f expref: -99 pid: 85754 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572152.390467:0:85754:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779c00/0xef30c340bfb44f3b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8317d8f expref: -99 pid: 85754 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:0.0:1788572152.392836:0:85754:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1788572152.392847:0:85754:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572152.392896:0:85754:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572152.392921:0:85754:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779c00/0xef30c340bfb44f3b lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8317d8f expref: -99 pid: 85754 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788572152.392946:0:85754:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779c00/0xef30c340bfb44f3b lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8317d8f expref: -99 pid: 85754 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572152.392955:0:85754:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572152.392964:0:85754:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572152.392968:0:85754:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb44f3b 00000080:00010000:0.0:1788572152.392976:0:85754:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb44f3b from it ffffab0ec1c57cb8 00010000:00010000:0.0:1788572152.392980:0:85754:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779c00/0xef30c340bfb44f3b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8317d8f expref: -99 pid: 85754 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788572152.392990:0:85754:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779c00/0xef30c340bfb44f3b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8317d8f expref: -99 pid: 85754 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788572152.393105:0:85754:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572152.393115:0:85754:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572152.393122:0:85754:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779c00/0xef30c340bfb44f3b lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8317d8f expref: -99 pid: 85754 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788572152.393132:0:85754:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779c00/0xef30c340bfb44f3b lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8317d8f expref: -99 pid: 85754 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788572152.393139:0:85754:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572152.393143:0:85754:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572152.393146:0:85754:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb44f3b 00000080:00010000:0.0:1788572152.393151:0:85754:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb44f3b from it ffffab0ec1c57bb0 00010000:00010000:0.0:1788572152.393154:0:85754:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779c00/0xef30c340bfb44f3b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8317d8f expref: -99 pid: 85754 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788572152.393163:0:85754:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779c00/0xef30c340bfb44f3b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8317d8f expref: -99 pid: 85754 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00000080:00200000:0.0:1788572152.393173:0:85754:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:0.0:1788572152.393180:0:85754:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200080:0.0:1788572152.393274:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.393289:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00800000:00000004:0.0:1788572152.393297:0:85754:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: age 4023, max_age 4069 00000100:00100000:0.0:1788572152.393340:0:85754:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37480 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:85754:1875449615037824:192.168.202.113@tcp:41:lfs.0 00000100:00100000:0.0:1788572152.393432:0:85754:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada455ea9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572152.397069:0:85754:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37480 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:85754:1875449615037824:192.168.202.113@tcp:41:lfs.0 00800000:00000004:0.0:1788572152.397100:0:85754:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: update ffff9ada4481d198 cache blocks 317317/353529 objects 1022888/1024000 00000080:00200080:0.0:1788572152.397182:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.397199:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00800000:00000004:0.0:1788572152.397208:0:85754:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: age 4023, max_age 4069 00000100:00100000:0.0:1788572152.397244:0:85754:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37480 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:85754:1875449615037952:192.168.202.113@tcp:41:lfs.0 00000100:00100000:0.0:1788572152.397313:0:85754:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada455ea9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572152.401371:0:85754:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37480 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:85754:1875449615037952:192.168.202.113@tcp:41:lfs.0 00800000:00000004:0.0:1788572152.401402:0:85754:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: update ffff9ada44819e18 cache blocks 317392/353529 objects 1023736/1024000 00000080:00200080:0.0:1788572152.401529:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401547:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401555:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401560:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401565:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401569:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401573:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401576:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401581:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401584:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401589:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401592:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401596:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401600:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401604:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401608:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401632:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401637:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401642:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401645:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401650:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401654:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401658:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401662:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401666:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401669:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401673:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401676:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401680:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401684:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401688:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401692:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401696:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401699:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401703:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401707:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401711:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401714:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401718:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401722:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401726:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401730:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401733:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401736:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401740:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401743:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401747:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401751:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401755:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401759:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401762:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401766:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401770:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401774:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401779:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401783:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401787:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401790:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401794:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401798:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401802:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401805:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401810:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401813:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401817:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401820:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401824:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401827:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401831:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401835:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401839:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401843:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401846:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401850:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401885:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401890:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401893:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401896:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401900:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401903:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401907:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401910:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401914:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401918:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401921:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401925:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401929:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401932:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401937:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401940:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401944:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401947:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401951:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401955:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401958:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401962:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401966:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401970:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401973:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401976:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401980:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401984:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401988:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.401991:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.401996:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.402000:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.402004:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.402008:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.402012:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.402016:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.402020:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.402024:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.402028:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.402031:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.402036:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.402039:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.402044:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.402047:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.402051:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.402055:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.402059:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.402063:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.402067:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.402071:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.402076:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00800000:00000080:0.0:1788572152.402079:0:85754:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.402084:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.402090:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00020000:00000004:0.0:1788572152.402100:0:85754:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: age 4023, max_age 4069 00000100:00100000:0.0:1788572152.402140:0:85754:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37480 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:85754:1875449615038080:192.168.202.113@tcp:13:lfs.0 00000100:00100000:0.0:1788572152.402220:0:85754:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada455ea9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572152.411294:0:85754:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37480 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:85754:1875449615038080:192.168.202.113@tcp:13:lfs.0 00020000:00000004:0.0:1788572152.411324:0:85754:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: update ffff9ada4354d198 cache blocks 901357/958279 objects 261345/262144 00000080:00200080:0.0:1788572152.411395:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.411415:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00020000:00000004:0.0:1788572152.411424:0:85754:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: age 4023, max_age 4069 00000100:00100000:0.0:1788572152.411459:0:85754:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37480 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:85754:1875449615038208:192.168.202.113@tcp:13:lfs.0 00000100:00100000:0.0:1788572152.411542:0:85754:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada455ea9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572152.415379:0:85754:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37480 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:85754:1875449615038208:192.168.202.113@tcp:13:lfs.0 00020000:00000004:0.0:1788572152.415412:0:85754:0:(obd_class.h:1116:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: update ffff9ada43549e18 cache blocks 901363/958279 objects 261350/262144 00000080:00200080:0.0:1788572152.415482:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415500:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415507:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415512:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415516:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415520:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415524:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415528:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415532:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415536:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415540:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415543:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415547:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415552:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415556:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415560:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415563:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415567:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415571:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415575:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415579:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415583:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415587:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415590:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415594:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415598:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415602:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415605:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415631:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415636:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415641:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415645:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415649:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415653:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415657:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415661:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415665:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415669:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415674:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415677:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415682:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415685:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415689:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415693:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415697:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415700:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415704:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415708:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415712:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415715:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415719:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415722:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415727:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415730:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415735:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415739:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415743:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415747:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415751:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415755:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415759:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415762:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415766:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415770:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415774:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415778:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415782:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415786:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415790:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415794:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415798:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415801:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415805:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415809:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415812:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415816:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415820:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415824:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415828:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415831:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415835:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415839:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415843:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415846:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415850:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415904:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415910:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415914:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415919:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415923:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415927:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415931:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415935:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415939:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415943:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415947:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415951:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415954:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415958:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415962:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415965:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415968:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415971:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415975:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415979:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415982:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415986:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415990:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.415994:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.415997:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.416002:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.416005:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.416010:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.416014:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.416018:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.416021:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.416025:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.416029:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.416033:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.416037:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.416041:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.416044:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.416049:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.416053:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200080:0.0:1788572152.416057:0:85754:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcd8396500 00020000:00000080:0.0:1788572152.416060:0:85754:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada60ae0000 uarg=0000000000000000 00000080:00200008:0.0:1788572152.416074:0:85754:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200000:0.0:1788572152.486145:0:85761:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572152.486176:0:85761:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572152.486196:0:85761:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779c00/0xef30c340bfb44f3b lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8317d8f expref: -99 pid: 85754 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00010000:00010000:0.0:1788572152.486213:0:85761:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779c00/0xef30c340bfb44f3b lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8317d8f expref: -99 pid: 85754 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00000080:00200000:0.0:1788572152.486222:0:85761:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572152.486229:0:85761:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572152.486233:0:85761:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb44f3b 00000080:00010000:0.0:1788572152.486241:0:85761:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb44f3b from it ffffab0ec1d1fbd0 00010000:00010000:0.0:1788572152.486245:0:85761:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779c00/0xef30c340bfb44f3b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8317d8f expref: -99 pid: 85754 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00010000:00010000:0.0:1788572152.486256:0:85761:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779c00/0xef30c340bfb44f3b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8317d8f expref: -99 pid: 85754 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00200000:0.0:1788572152.486273:0:85761:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:0.0:1788572152.486286:0:85761:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=fsa-oleg213-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), flags=1536 00000080:00200000:0.0:1788572152.486290:0:85761:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572152.486297:0:85761:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572152.486304:0:85761:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779c00/0xef30c340bfb44f3b lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8317d8f expref: -99 pid: 85754 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:0.0:1788572152.486312:0:85761:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779c00/0xef30c340bfb44f3b lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8317d8f expref: -99 pid: 85754 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00200000:0.0:1788572152.486319:0:85761:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572152.486323:0:85761:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572152.486326:0:85761:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb44f3b 00000080:00010000:0.0:1788572152.486330:0:85761:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb44f3b from it ffffab0ec1d1fcd0 00010000:00010000:0.0:1788572152.486334:0:85761:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779c00/0xef30c340bfb44f3b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8317d8f expref: -99 pid: 85754 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:0.0:1788572152.486342:0:85761:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779c00/0xef30c340bfb44f3b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8317d8f expref: -99 pid: 85754 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:0.0:1788572152.486349:0:85761:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:0.0:1788572152.486355:0:85761:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572152.486361:0:85761:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572152.486368:0:85761:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779c00/0xef30c340bfb44f3b lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8317d8f expref: -99 pid: 85754 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:0.0:1788572152.486376:0:85761:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779c00/0xef30c340bfb44f3b lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8317d8f expref: -99 pid: 85754 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:0.0:1788572152.486384:0:85761:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572152.486387:0:85761:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572152.486390:0:85761:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb44f3b 00000080:00010000:0.0:1788572152.486394:0:85761:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb44f3b from it ffffab0ec1d1fdb0 00010000:00010000:0.0:1788572152.486397:0:85761:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779c00/0xef30c340bfb44f3b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8317d8f expref: -99 pid: 85754 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:0.0:1788572152.486405:0:85761:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779c00/0xef30c340bfb44f3b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8317d8f expref: -99 pid: 85754 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000080:00200000:0.0:1788572152.486414:0:85761:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:0.0:1788572152.486420:0:85761:0:(namei.c:2062:ll_create_nd()) VFS Op:name=fsa-oleg213-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), flags=33188, excl=1 00000080:00200000:0.0:1788572152.486424:0:85761:0:(namei.c:2017:ll_mknod()) VFS Op:name=fsa-oleg213-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9ada58125b88) mode 100644 dev 0 00010000:00010000:0.0:1788572152.486451:0:85761:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779c00/0xef30c340bfb44f3b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b8317d8f expref: -99 pid: 85754 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:0.0:1788572152.486628:0:85761:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779c00/0xef30c340bfb44f3b lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b8317d8f expref: -99 pid: 85754 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:0.0:1788572152.486636:0:85761:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1788572152.486640:0:85761:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779c00/0xef30c340bfb44f3b lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b8317d8f expref: -99 pid: 85754 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000020:00100000:0.0:1788572152.486671:0:85761:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1788572152.486686:0:85761:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada60a37480 x1875449615038464/t0(0) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788572152.486711:0:85761:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37480 pname:cluuid:pid:xid:nid:opc:job mcreate:35eb2d60-2de0-4b00-9815-d5153edd0057:85761:1875449615038464:192.168.202.113@tcp:36:mcreate.0 00000100:00100000:0.0:1788572152.486804:0:85761:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada455ea9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572152.505057:0:85761:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37480 pname:cluuid:pid:xid:nid:opc:job mcreate:35eb2d60-2de0-4b00-9815-d5153edd0057:85761:1875449615038464:192.168.202.113@tcp:36:mcreate.0 00000080:00200000:2.0:1788572152.505426:0:85761:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x2000182d1:0x642:0x0](ffff9ada46139988) 00000080:00200000:2.0:1788572152.505472:0:85761:0:(namei.c:2070:ll_create_nd()) VFS Op:name=fsa-oleg213-client.virtnet, unhashed 1 00000080:00200000:1.0:1788572152.524453:0:85764:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:1.0:1788572152.524489:0:85764:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1788572152.524561:0:85764:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467000/0xef30c340bfb44f42 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 85764 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572152.524571:0:85764:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467000/0xef30c340bfb44f42 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 85764 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572152.524580:0:85764:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467000/0xef30c340bfb44f42 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 85764 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572152.524616:0:85764:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516b4700 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:85764:1875449615038592:192.168.202.113@tcp:101:rm.0 00000100:00100000:1.0:1788572152.524714:0:85764:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada605e9c00 going to sleep for 11 seconds 00000100:00100000:1.0:1788572152.528152:0:85764:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516b4700 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:85764:1875449615038592:192.168.202.113@tcp:101:rm.0 00010000:00010000:1.0:1788572152.528185:0:85764:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467000/0xef30c340bfb44f42 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8317dab expref: -99 pid: 85764 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572152.528198:0:85764:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467000/0xef30c340bfb44f42 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8317dab expref: -99 pid: 85764 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572152.528209:0:85764:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467000/0xef30c340bfb44f42 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8317dab expref: -99 pid: 85764 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572152.528289:0:85764:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467000/0xef30c340bfb44f42 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8317dab expref: -99 pid: 85764 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572152.528305:0:85764:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467000/0xef30c340bfb44f42 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8317dab expref: -99 pid: 85764 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572152.528314:0:85764:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467000/0xef30c340bfb44f42 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8317dab expref: -99 pid: 85764 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572152.528322:0:85764:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467000/0xef30c340bfb44f42 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8317dab expref: -99 pid: 85764 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1788572152.528334:0:85764:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada516b4700 x1875449615038592/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788572168 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1788572152.528348:0:85764:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467000/0xef30c340bfb44f42 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8317dab expref: -99 pid: 85764 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572152.528362:0:85764:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1788572152.528368:0:85764:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1788572152.528396:0:85764:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:1.0:1788572152.528401:0:85764:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb44f42 00000080:00010000:1.0:1788572152.528408:0:85764:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:1.0:1788572152.528411:0:85764:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb44f42 from it ffffab0ec1cdfb58 00010000:00010000:1.0:1788572152.528414:0:85764:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467000/0xef30c340bfb44f42 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8317dab expref: -99 pid: 85764 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572152.528428:0:85764:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467000/0xef30c340bfb44f42 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8317dab expref: -99 pid: 85764 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572152.528450:0:85764:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:1.0:1788572152.528470:0:85764:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=fsa-oleg213-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), flags=8 00000080:00200000:1.0:1788572152.528474:0:85764:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=fsa-oleg213-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=getattr 00000002:00010000:1.0:1788572152.528495:0:85764:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: fsa-oleg213-client.virtnet,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 01 00010000:00010000:1.0:1788572152.528532:0:85764:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467e00/0xef30c340bfb44f49 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 85764 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572152.528540:0:85764:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467e00/0xef30c340bfb44f49 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 85764 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572152.528550:0:85764:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467e00/0xef30c340bfb44f49 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 85764 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572152.528570:0:85764:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516b4700 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:85764:1875449615038720:192.168.202.113@tcp:101:rm.0 00000100:00100000:1.0:1788572152.530117:0:85764:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada605e9c00 going to sleep for 11 seconds 00000100:00100000:1.0:1788572152.544630:0:85764:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516b4700 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:85764:1875449615038720:192.168.202.113@tcp:101:rm.0 00010000:00010000:1.0:1788572152.544659:0:85764:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467e00/0xef30c340bfb44f49 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8317dc7 expref: -99 pid: 85764 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572152.544683:0:85764:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467e00/0xef30c340bfb44f49 lrc: 4/1,0 mode: --/PR res: [0x2000182d1:0x642:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8317dc7 expref: -99 pid: 85764 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572152.544691:0:85764:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467e00/0xef30c340bfb44f49 lrc: 4/1,0 mode: --/PR res: [0x2000182d1:0x642:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8317dc7 expref: -99 pid: 85764 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572152.544703:0:85764:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467e00/0xef30c340bfb44f49 lrc: 4/1,0 mode: PR/PR res: [0x2000182d1:0x642:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8317dc7 expref: -99 pid: 85764 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572152.544714:0:85764:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467e00/0xef30c340bfb44f49 lrc: 4/1,0 mode: PR/PR res: [0x2000182d1:0x642:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8317dc7 expref: -99 pid: 85764 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572152.544726:0:85764:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467e00/0xef30c340bfb44f49 lrc: 5/2,0 mode: PR/PR res: [0x2000182d1:0x642:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8317dc7 expref: -99 pid: 85764 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572152.544733:0:85764:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467e00/0xef30c340bfb44f49 lrc: 5/2,0 mode: PR/PR res: [0x2000182d1:0x642:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8317dc7 expref: -99 pid: 85764 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572152.544740:0:85764:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467e00/0xef30c340bfb44f49 lrc: 4/1,0 mode: PR/PR res: [0x2000182d1:0x642:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8317dc7 expref: -99 pid: 85764 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1788572152.544751:0:85764:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada516b4700 x1875449615038720/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 600/584 e 0 to 0 dl 1788572168 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1788572152.544763:0:85764:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467e00/0xef30c340bfb44f49 lrc: 3/1,0 mode: PR/PR res: [0x2000182d1:0x642:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8317dc7 expref: -99 pid: 85764 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572152.544773:0:85764:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8590033617/1602 (0 0) 00000080:00200000:1.0:1788572152.544778:0:85764:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1788572152.544797:0:85764:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x2000182d1:0x642:0x0], updating i_size 0 00000080:00200000:1.0:1788572152.544810:0:85764:0:(namei.c:147:ll_iget()) got inode: [0x2000182d1:0x642:0x0](ffff9ada46139988): rc = 0 00000080:00200000:1.0:1788572152.544827:0:85764:0:(file.c:6843:ll_layout_conf()) [0x2000182d1:0x642:0x0]: layout version change: 4294967294 -> 4294967295 00000080:00010000:1.0:1788572152.544833:0:85764:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x2000182d1:0x642:0x0](ffff9ada46139988) for lock 0xef30c340bfb44f49 00000080:00010000:1.0:1788572152.544844:0:85764:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x2000182d1:0x642:0x0](ffff9ada46139988) 00000080:00010000:1.0:1788572152.544848:0:85764:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb44f49 from it ffffab0ec1cdfb20 00010000:00010000:1.0:1788572152.544851:0:85764:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467e00/0xef30c340bfb44f49 lrc: 3/1,0 mode: PR/PR res: [0x2000182d1:0x642:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b8317dc7 expref: -99 pid: 85764 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572152.545066:0:85764:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467e00/0xef30c340bfb44f49 lrc: 2/0,0 mode: PR/PR res: [0x2000182d1:0x642:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b8317dc7 expref: -99 pid: 85764 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:1.0:1788572152.545103:0:85764:0:(file.c:6296:ll_getattr_dentry()) START file fsa-oleg213-client.virtnet:[0x2000182d1:0x642:0x0](ffff9ada46139988), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1788572152.545109:0:85764:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x2000182d1:0x642:0x0](ffff9ada46139988),name=fsa-oleg213-client.virtnet 00000002:00010000:1.0:1788572152.545130:0:85764:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x2000182d1:0x642:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1788572152.545140:0:85764:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467e00/0xef30c340bfb44f49 lrc: 2/1,0 mode: PR/PR res: [0x2000182d1:0x642:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b8317dc7 expref: -99 pid: 85764 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1788572152.545152:0:85764:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467e00/0xef30c340bfb44f49 lrc: 2/1,0 mode: PR/PR res: [0x2000182d1:0x642:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b8317dc7 expref: -99 pid: 85764 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572152.545158:0:85764:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1788572152.545162:0:85764:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x2000182d1:0x642:0x0](ffff9ada46139988) 00000080:00010000:1.0:1788572152.545165:0:85764:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x2000182d1:0x642:0x0](ffff9ada46139988) for lock 0xef30c340bfb44f49 00000080:00010000:1.0:1788572152.545169:0:85764:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb44f49 from it ffffab0ec1cdfcb8 00010000:00010000:1.0:1788572152.545172:0:85764:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467e00/0xef30c340bfb44f49 lrc: 3/1,0 mode: PR/PR res: [0x2000182d1:0x642:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b8317dc7 expref: -99 pid: 85764 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1788572152.545179:0:85764:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467e00/0xef30c340bfb44f49 lrc: 2/0,0 mode: PR/PR res: [0x2000182d1:0x642:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b8317dc7 expref: -99 pid: 85764 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1788572152.546557:0:85764:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:1.0:1788572152.546580:0:85764:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1788572152.546593:0:85764:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467000/0xef30c340bfb44f42 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8317dab expref: -99 pid: 85764 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1788572152.546616:0:85764:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467000/0xef30c340bfb44f42 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8317dab expref: -99 pid: 85764 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572152.546624:0:85764:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1788572152.546631:0:85764:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:1.0:1788572152.546634:0:85764:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb44f42 00000080:00010000:1.0:1788572152.546640:0:85764:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb44f42 from it ffffab0ec1cdfc10 00010000:00010000:1.0:1788572152.546643:0:85764:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467000/0xef30c340bfb44f42 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8317dab expref: -99 pid: 85764 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1788572152.546651:0:85764:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467000/0xef30c340bfb44f42 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8317dab expref: -99 pid: 85764 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1788572152.546662:0:85764:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:1.0:1788572152.546672:0:85764:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=fsa-oleg213-client.virtnet, flags=0 00000080:00200000:1.0:1788572152.546677:0:85764:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:1.0:1788572152.546684:0:85764:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1788572152.546691:0:85764:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467000/0xef30c340bfb44f42 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8317dab expref: -99 pid: 85764 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1788572152.546698:0:85764:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467000/0xef30c340bfb44f42 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8317dab expref: -99 pid: 85764 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1788572152.546706:0:85764:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1788572152.546710:0:85764:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:1.0:1788572152.546713:0:85764:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb44f42 00000080:00010000:1.0:1788572152.546716:0:85764:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb44f42 from it ffffab0ec1cdfd68 00010000:00010000:1.0:1788572152.546719:0:85764:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467000/0xef30c340bfb44f42 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8317dab expref: -99 pid: 85764 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1788572152.546727:0:85764:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467000/0xef30c340bfb44f42 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8317dab expref: -99 pid: 85764 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1788572152.546735:0:85764:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:1.0:1788572152.546741:0:85764:0:(namei.c:2351:ll_unlink()) VFS Op:name=fsa-oleg213-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9ada58125b88) 00010000:00010000:1.0:1788572152.546767:0:85764:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467000/0xef30c340bfb44f42 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b8317dab expref: -99 pid: 85764 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1788572152.547427:0:85764:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467e00/0xef30c340bfb44f49 lrc: 2/0,0 mode: PR/PR res: [0x2000182d1:0x642:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0xdb7edb52b8317dc7 expref: -99 pid: 85764 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1788572152.547443:0:85764:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x642:0x0]: losing layout lock 00010000:00010000:1.0:1788572152.547481:0:85764:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467e00/0xef30c340bfb44f49 lrc: 1/0,0 mode: --/PR res: [0x2000182d1:0x642:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0xdb7edb52b8317dc7 expref: -99 pid: 85764 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1788572152.547489:0:85764:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467000/0xef30c340bfb44f42 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b8317dab expref: -99 pid: 85764 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1788572152.547494:0:85764:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:1.0:1788572152.547497:0:85764:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467e00/0xef30c340bfb44f49 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x642:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0xdb7edb52b8317dc7 expref: -99 pid: 85764 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1788572152.547516:0:85764:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467000/0xef30c340bfb44f42 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b8317dab expref: -99 pid: 85764 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:1.0:1788572152.547534:0:85764:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1788572152.547545:0:85764:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada516b4700 x1875449615038976/t0(0) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 512/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788572152.547569:0:85764:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516b4700 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:85764:1875449615038976:192.168.202.113@tcp:36:rm.0 00000100:00100000:1.0:1788572152.547733:0:85764:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada605e9c00 going to sleep for 11 seconds 00000100:00100000:1.0:1788572152.557758:0:85764:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516b4700 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:85764:1875449615038976:192.168.202.113@tcp:36:rm.0 00000080:00200000:1.0:1788572152.557837:0:85764:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x642:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1788572152.557848:0:85764:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x642:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1788572152.557889:0:85764:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x642:0x0](ffff9ada46139988) 00010000:00010000:1.0:1788572152.697717:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572153.741160:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572154.747742:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572155.767324:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572156.793898:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572157.303397:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449615039104:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572157.313352:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1788572157.313415:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615039232:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572157.323753:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a36680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449615039488:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572157.323948:0:2383:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1788572157.323958:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449615039360:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572157.323989:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449615039616:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572157.362066:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449615039104:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572157.379977:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a35c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615039232:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572157.381040:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449615039360:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572157.381067:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a35880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449615039616:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572157.381079:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a36680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449615039488:192.168.202.113@tcp:400:kworker.0 00010000:00010000:0.0:1788572157.816311:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572158.839539:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572159.866743:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1788572160.800108:0:85831:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: local REPLAY BARRIER on lustre-MDT0000 00010000:00010000:3.0:1788572160.887071:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572161.919414:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788572162.431905:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517a1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449615039744:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572162.434833:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1788572162.434890:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517a3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615039872:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572162.439013:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516b5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449615040000:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572162.439187:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516b5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615040128:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572162.439251:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516b7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449615040256:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572162.448447:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516b5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615040128:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572162.448447:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517a3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615039872:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572162.448468:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517a1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449615039744:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572162.448561:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516b7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449615040256:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572162.448574:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516b5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449615040000:192.168.202.113@tcp:400:kworker.0 00010000:00010000:3.0:1788572162.936092:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572170.103293:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572171.132375:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572172.151888:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572172.663315:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60742d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449615194496:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572172.663447:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60536d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615194752:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572172.663481:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60537b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615195008:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572172.663511:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1788572172.663517:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60741880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615194624:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572172.663541:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60536300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615194880:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572172.674552:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60741880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615194624:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572172.674571:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60742d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449615194496:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572172.674583:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60536d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615194752:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572172.676182:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60536300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615194880:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572172.676199:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60537b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615195008:192.168.202.113@tcp:400:kworker.0 00010000:00010000:0.0:1788572173.175101:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572174.199269:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572175.225340:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572176.247143:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572177.271381:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788572177.783483:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517a0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449615195136:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572177.783582:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517a2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615195392:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572177.783605:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517a0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615195648:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572177.783633:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1788572177.783639:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517a2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615195264:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572177.783662:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517a1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615195520:192.168.202.113@tcp:400:kworker.0 00000100:00080000:3.0:1788572177.797141:0:2386:0:(recover.c:205:ptlrpc_request_handle_notconn()) import lustre-MDT0000-mdc-ffff9ada58b0a000 of lustre-MDT0000_UUID@192.168.202.113@tcp abruptly disconnected: reconnecting 00000100:00080000:3.0:1788572177.797152:0:2386:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:3.0:1788572177.797158:0:2386:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff9ada58b0a000: Connection to lustre-MDT0000 (at 192.168.202.113@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:3.0:1788572177.842636:0:2386:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572177.842657:0:2386:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4003 00000100:00080000:3.0:1788572177.843541:0:2386:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572177.843558:0:2386:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572177.843601:0:2386:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada517a0000 x1875449615195776/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788572177.843644:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517a2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615195264:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572177.843701:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517a1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615195520:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572177.843717:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517a0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449615195136:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572177.843733:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517a2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615195392:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572177.843746:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517a0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615195648:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572177.843931:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615195776:192.168.202.113@tcp:38:ptlrpcd_01_01.0 00000100:00080000:1.0:1788572177.849045:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572177.849075:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-19) 00000100:00100000:1.0:1788572177.849080:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615195776:192.168.202.113@tcp:38:ptlrpcd_01_01.0 00010000:00010000:3.0:1788572178.295582:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572179.320092:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572180.343885:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572181.370340:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572182.391474:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788572182.903213:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572182.903221:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788572182.903226:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572182.903239:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4095 00000100:00080000:1.0:1788572182.903253:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 5s 00000100:00080000:1.0:1788572182.903264:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572182.903279:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572182.903303:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60742d80 x1875449615196032/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788572182.903396:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60741880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615196160:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572182.903516:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada44fea300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615196416:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572182.903554:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1788572182.903560:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45534000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449615195904:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572182.903590:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada44fead80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615196288:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572182.904417:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60742d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615196032:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572182.906503:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60741880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615196160:192.168.202.113@tcp:400:kworker.0 00000100:00080000:2.0:1788572182.912447:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788572182.912467:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-19) 00000100:00100000:2.0:1788572182.912474:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60742d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615196032:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572182.917432:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada44fea300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615196416:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572182.922533:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada44fead80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615196288:192.168.202.113@tcp:400:kworker.0 00010000:00010000:0.0:1788572183.415612:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572184.439188:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572185.466636:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572186.488575:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572187.511165:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788572188.023292:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572188.023300:0:58077:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788572188.023305:0:58077:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572188.023318:0:58077:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4100 00000100:00080000:3.0:1788572188.023334:0:58077:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 10s 00000100:00080000:3.0:1788572188.023342:0:58077:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572188.023356:0:58077:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572188.023379:0:58077:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada517a2a00 x1875449615196672/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788572188.023464:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517a0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615196800:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572188.023569:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517a2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615197056:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572188.023604:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1788572188.023611:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517a0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449615196544:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572188.023643:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517a1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615196928:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572188.024361:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517a2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615196672:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572188.031421:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517a1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615196928:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572188.031454:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517a2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615197056:192.168.202.113@tcp:400:kworker.0 00000100:00080000:1.0:1788572188.031839:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572188.031848:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-19) 00000100:00100000:1.0:1788572188.031888:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517a2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615196672:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572188.032252:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517a0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615196800:192.168.202.113@tcp:400:kworker.0 00010000:00010000:0.0:1788572188.535553:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572189.563241:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572190.588232:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572191.607379:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572192.635345:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788572193.143151:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572193.143161:0:58077:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788572193.143167:0:58077:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572193.143180:0:58077:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4105 00000100:00080000:2.0:1788572193.143194:0:58077:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 15s 00000100:00080000:2.0:1788572193.143204:0:58077:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572193.143218:0:58077:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572193.143230:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1788572193.143240:0:58077:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada517fd880 x1875449615197312/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788572193.143255:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517fd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449615197184:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572193.143373:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615197440:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572193.143419:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517fe680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615197696:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572193.143457:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1788572193.143464:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615197568:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572193.143488:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615197312:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572193.147710:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615197568:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572193.147732:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615197440:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572193.147742:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517fe680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615197696:192.168.202.113@tcp:400:kworker.0 00000100:00080000:1.0:1788572193.148123:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572193.148131:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-19) 00000100:00100000:1.0:1788572193.148136:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615197312:192.168.202.113@tcp:38:kworker.0 00010000:00010000:3.0:1788572193.657273:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572194.679531:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572195.703804:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572196.728683:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572197.751461:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788572198.263120:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572198.263130:0:58077:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788572198.263134:0:58077:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572198.263147:0:58077:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4110 00000100:00080000:2.0:1788572198.263159:0:58077:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572198.263167:0:58077:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572198.263182:0:58077:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788572198.263215:0:58077:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada517fd180 x1875449615197952/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788572198.263292:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517fd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615197952:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788572198.263500:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615198080:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572198.263540:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615198336:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572198.263575:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1788572198.263582:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517fc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449615197824:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572198.263627:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517fc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615198208:192.168.202.113@tcp:400:kworker.0 00000100:00000400:0.0:1788572198.266461:0:2384:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788572182/real 1788572182] req@ffff9ada45534000 x1875449615195904/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 1 dl 1788572198 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572198.266501:0:2384:0:(client.c:2548:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9ada45534000 x1875449615195904/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 1 dl 1788572198 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572198.266511:0:2384:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from FULL to DISCONN 00000100:02020000:0.0:1788572198.266516:0:2384:0:(import.c:186:ptlrpc_set_import_discon()) MGC192.168.202.113@tcp: Connection to MGS (at 192.168.202.113@tcp) was lost; in progress operations using this service will fail 00000100:00080000:0.0:1788572198.266519:0:2384:0:(import.c:192:ptlrpc_set_import_discon()) import MGS@192.168.202.113@tcp for MGC192.168.202.113@tcp not replayable, auto-deactivating 00000100:00080000:0.0:1788572198.266522:0:2384:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import MGS INVALID 00000100:00100000:0.0:1788572198.266526:0:2384:0:(client.c:3591:ptlrpc_abort_inflight()) @@@ inflight req@ffff9ada45534000 x1875449615195904/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 1 dl 1788572198 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572198.266536:0:2384:0:(client.c:3591:ptlrpc_abort_inflight()) @@@ inflight req@ffff9ada517a0a80 x1875449615196544/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572204 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572198.266562:0:2384:0:(client.c:3591:ptlrpc_abort_inflight()) @@@ inflight req@ffff9ada517fd500 x1875449615197184/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572209 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572198.266570:0:2384:0:(client.c:3591:ptlrpc_abort_inflight()) @@@ inflight req@ffff9ada517fc700 x1875449615197824/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572214 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:0.0:1788572198.266580:0:2384:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808001 00000100:00080000:0.0:1788572198.266583:0:2384:0:(pinger.c:193:ptlrpc_pinger_ir_down()) IR down 10000000:01000000:0.0:1788572198.266585:0:2384:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808002 00000100:00080000:0.0:1788572198.266587:0:2384:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00100000:0.0:1788572198.266621:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45534000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449615195904:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572198.267950:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517fc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449615197824:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572198.267993:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615198080:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572198.268004:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517fd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449615197184:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572198.268011:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517a0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449615196544:192.168.202.113@tcp:400:kworker.0 00000100:00080000:2.0:1788572198.268064:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572198.268070:0:58077:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788572198.268074:0:58077:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572198.268080:0:58077:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4002 00000100:00080000:2.0:1788572198.268092:0:58077:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788572198.268126:0:58077:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada517fd500 x1875449615198464/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788572198.269414:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517fd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449615198464:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788572198.272890:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572198.272912:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-19) 00000100:00100000:1.0:1788572198.272919:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517fd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615197952:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788572198.273125:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615198336:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572198.273147:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517fc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615198208:192.168.202.113@tcp:400:kworker.0 00010000:00010000:0.0:1788572198.778464:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572199.799150:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572200.823216:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572201.848506:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572202.884328:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788572203.383637:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572203.383652:0:58077:0:(pinger.c:247:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1788572203.383655:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572203.383659:0:58077:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788572203.383665:0:58077:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572203.383688:0:58077:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4115 00000100:00080000:2.0:1788572203.383702:0:58077:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572203.383712:0:58077:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572203.383728:0:58077:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788572203.383778:0:58077:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada517fc000 x1875449615198592/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788572203.383934:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615198848:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572203.384085:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1788572203.384094:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615198720:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572203.384127:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517fc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615198976:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572203.386052:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517fc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615198592:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788572203.391924:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615198720:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572203.391953:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517fc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615198976:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572203.391968:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615198848:192.168.202.113@tcp:400:kworker.0 00000100:00080000:1.0:1788572203.394698:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572203.394721:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-19) 00000100:00100000:1.0:1788572203.394728:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517fc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615198592:192.168.202.113@tcp:38:kworker.0 00010000:00010000:0.0:1788572203.896207:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572204.925221:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572205.945239:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572206.967345:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572207.991243:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572208.503089:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788572198/real 1788572198] req@ffff9ada517fd500 x1875449615198464/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572208 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572208.503139:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada517fd500 x1875449615198464/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572208 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572208.503166:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572208.503200:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572208.503206:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517fd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449615198464:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788572208.503652:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572208.503659:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788572208.503663:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572208.503670:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4115 00000100:00080000:0.0:1788572208.503754:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 5s 00000100:00080000:0.0:1788572208.503764:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788572208.503815:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60535c00 x1875449615199104/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788572208.503938:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572208.503943:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788572208.503944:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572208.503949:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4121 00000100:00080000:0.0:1788572208.503954:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572208.503958:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788572208.503970:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788572208.503984:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60534000 x1875449615199232/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788572208.504054:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60537800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449615199360:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572208.504227:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60535f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449615199616:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572208.504294:0:2383:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1788572208.504301:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60534a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449615199488:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572208.506921:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60535c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449615199104:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788572208.507053:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60534000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615199232:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788572208.516379:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60534a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449615199488:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572208.516406:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60537800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449615199360:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572208.516416:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60535f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449615199616:192.168.202.113@tcp:400:kworker.0 00000100:00080000:1.0:1788572208.524646:0:2382:0:(import.c:1176:ptlrpc_connect_interpret()) MGC192.168.202.113@tcp: connect to target with instance 0 10000000:01000000:1.0:1788572208.524671:0:2382:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808005 00000100:02000400:1.0:1788572208.524749:0:2382:0:(import.c:1328:ptlrpc_connect_interpret()) Evicted from MGS (at 192.168.202.113@tcp) after server handle changed from 0xdb7edb52b83068de to 0xdb7edb52b831c954 00000100:00080000:1.0:1788572208.524757:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to EVICTED 00000100:00080000:1.0:1788572208.524765:0:2382:0:(import.c:1718:ptlrpc_import_recovery_state_machine()) evicted from MGS@192.168.202.113@tcp; invalidating 00000100:00100000:1.0:1788572208.525597:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60535c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449615199104:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788572208.525631:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572208.525637:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-19) 00000100:00100000:1.0:1788572208.525641:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60534000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615199232:192.168.202.113@tcp:38:kworker.0 00000100:00080000:3.0:1788572208.535163:0:86247:0:(import.c:1642:ptlrpc_invalidate_import_thread()) thread invalidate import MGC192.168.202.113@tcp to MGS@192.168.202.113@tcp 00000100:00100000:3.0:1788572208.535174:0:86247:0:(import.c:323:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 10000000:01000000:3.0:1788572208.535184:0:86247:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808003 00010000:00010000:3.0:1788572208.535212:0:86247:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.202.113@tcp lock: ffff9ada45519e00/0xef30c340bfb40d7f lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x110e400000000 nid: local remote: 0xdb7edb52b8306a27 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1788572208.535228:0:86247:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.202.113@tcp lock: ffff9ada45519e00/0xef30c340bfb40d7f lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x110f400000000 nid: local remote: 0xdb7edb52b8306a27 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1788572208.535235:0:86247:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:3.0:1788572208.535275:0:86247:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1788572208.535292:0:86247:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.202.113@tcp lock: ffff9ada45519e00/0xef30c340bfb40d7f lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x590f400000000 nid: local remote: 0xdb7edb52b8306a27 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572208.535313:0:86247:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.202.113@tcp lock: ffff9ada607bd400/0xef30c340bfb40d8d lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0xdb7edb52b8306a7b expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1788572208.535321:0:86247:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.202.113@tcp lock: ffff9ada607bd400/0xef30c340bfb40d8d lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0xdb7edb52b8306a7b expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1788572208.535332:0:86247:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:3.0:1788572208.535336:0:86247:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1788572208.535340:0:86247:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.202.113@tcp lock: ffff9ada607bd400/0xef30c340bfb40d8d lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0xdb7edb52b8306a7b expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572208.535362:0:86247:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.202.113@tcp lock: ffff9ada607bd000/0xef30c340bfb40d78 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0xdb7edb52b8306a04 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1788572208.535745:0:86247:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.202.113@tcp lock: ffff9ada607bd000/0xef30c340bfb40d78 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0xdb7edb52b8306a04 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1788572208.535751:0:86247:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:2.0:1788572208.535980:0:86247:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788572208.535988:0:86247:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.202.113@tcp lock: ffff9ada607bd000/0xef30c340bfb40d78 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0xdb7edb52b8306a04 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572208.536011:0:86247:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.202.113@tcp lock: ffff9ada607bc600/0xef30c340bfb40d86 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0xdb7edb52b8306a58 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1788572208.536019:0:86247:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.202.113@tcp lock: ffff9ada607bc600/0xef30c340bfb40d86 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0xdb7edb52b8306a58 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1788572208.536024:0:86247:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:2.0:1788572208.536029:0:86247:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1788572208.536032:0:86247:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.202.113@tcp lock: ffff9ada607bc600/0xef30c340bfb40d86 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0xdb7edb52b8306a58 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1788572208.536052:0:86247:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from EVICTED to RECOVER 00000100:00080000:2.0:1788572208.536058:0:86247:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from RECOVER to FULL 10000000:01000000:2.0:1788572208.536062:0:86247:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808004 00000100:00080000:2.0:1788572208.536065:0:86247:0:(pinger.c:186:ptlrpc_pinger_ir_up()) IR up 00000100:02000000:2.0:1788572208.536071:0:86247:0:(import.c:1787:ptlrpc_import_recovery_state_machine()) MGC192.168.202.113@tcp: Connection restored to 192.168.202.113@tcp (at 192.168.202.113@tcp) 00010000:00010000:3.0:1788572209.015882:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1788572209.591122:0:15207:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:1.0:1788572209.591133:0:15207:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9ada58b0a000 from 204 10000000:01000000:1.0:1788572209.591148:0:15207:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:1.0:1788572209.591240:0:15207:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.202.113@tcp lock: ffff9ada4542aa00/0xef30c340bfb45a40 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572209.591254:0:15207:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.202.113@tcp lock: ffff9ada4542aa00/0xef30c340bfb45a40 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572209.591267:0:15207:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.202.113@tcp lock: ffff9ada4542aa00/0xef30c340bfb45a40 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572209.591303:0:15207:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada44fead80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449615199744:192.168.202.113@tcp:101:ll_cfg_requeue.0 00000100:00100000:1.0:1788572209.591435:0:15207:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60703d80 going to sleep for 11 seconds 00000100:00100000:1.0:1788572209.597149:0:15207:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada44fead80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449615199744:192.168.202.113@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1788572209.597263:0:15207:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.202.113@tcp lock: ffff9ada4542aa00/0xef30c340bfb45a40 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b831ca7a expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572209.597280:0:15207:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.202.113@tcp lock: ffff9ada4542aa00/0xef30c340bfb45a40 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0xdb7edb52b831ca7a expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572209.597378:0:15207:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada44fead80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449615199872:192.168.202.113@tcp:256:ll_cfg_requeue.0 00000100:00100000:1.0:1788572209.597549:0:15207:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60703d80 going to sleep for 11 seconds 00000100:00100000:1.0:1788572209.605426:0:15207:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada44fead80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449615199872:192.168.202.113@tcp:256:ll_cfg_requeue.0 00000100:00080000:1.0:1788572209.605505:0:15207:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff9ada51140000 already not connected (conn 341, was 0): DISCONN 00010000:00080000:1.0:1788572209.605523:0:15207:0:(ldlm_lib.c:85:import_set_conn()) imp ffff9ada51140000@lustre-MDT0000-mdc-ffff9ada58b0a000: found existing conn 192.168.202.113@tcp, moved to head 00000100:00080000:1.0:1788572209.605530:0:15207:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572209.605534:0:15207:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 0 00000100:00080000:1.0:1788572209.605599:0:15207:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572209.605613:0:15207:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572209.605649:0:15207:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada44fea300 x1875449615200000/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788572209.605738:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada44fea300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615200000:192.168.202.113@tcp:38:ll_cfg_requeue.0 10000000:01000000:1.0:1788572209.606449:0:15207:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.202.113@tcp: configuration from log 'lustre-cliir' succeeded (0). 00010000:00010000:1.0:1788572209.606463:0:15207:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.202.113@tcp lock: ffff9ada4542aa00/0xef30c340bfb45a40 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b831ca7a expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572209.606476:0:15207:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.202.113@tcp lock: ffff9ada4542aa00/0xef30c340bfb45a40 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b831ca7a expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1788572209.606486:0:15207:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:1.0:1788572209.606489:0:15207:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9ada58b0a000 from 52 10000000:01000000:1.0:1788572209.606492:0:15207:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:1.0:1788572209.606524:0:15207:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.202.113@tcp lock: ffff9ada4542b400/0xef30c340bfb45a47 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572209.606533:0:15207:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.202.113@tcp lock: ffff9ada4542b400/0xef30c340bfb45a47 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572209.606609:0:15207:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.202.113@tcp lock: ffff9ada4542b400/0xef30c340bfb45a47 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572209.606630:0:15207:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada44fead80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449615200128:192.168.202.113@tcp:101:ll_cfg_requeue.0 00000100:00100000:1.0:1788572209.606727:0:15207:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60703d80 going to sleep for 11 seconds 00000100:00100000:1.0:1788572209.609944:0:15207:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada44fead80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449615200128:192.168.202.113@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1788572209.609984:0:15207:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.202.113@tcp lock: ffff9ada4542b400/0xef30c340bfb45a47 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b831ca81 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572209.610000:0:15207:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.202.113@tcp lock: ffff9ada4542b400/0xef30c340bfb45a47 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0xdb7edb52b831ca81 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572209.610094:0:15207:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada44fead80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449615200256:192.168.202.113@tcp:501:ll_cfg_requeue.0 00000100:00100000:1.0:1788572209.611308:0:15207:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60703d80 going to sleep for 11 seconds 00010000:00080000:0.0:1788572209.611720:0:2382:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff9ada44fea300 x1875449615200000/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/416 e 0 to 0 dl 1788572219 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572209.611746:0:2382:0:(import.c:1176:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to target with instance 51 00000100:00080000:0.0:1788572209.611771:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to REPLAY 00000100:00080000:0.0:1788572209.611964:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572209.611968:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572209.611976:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572168 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572209.611986:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 0 00000100:00080000:0.0:1788572209.611995:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572168 ref 1 fl New:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572209.612009:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada44fea300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615200000:192.168.202.113@tcp:38:ll_cfg_requeue.0 00000100:00100000:0.0:1788572209.612041:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615038464:192.168.202.113@tcp:36:mcreate.0 00000100:00100000:1.0:1788572209.618813:0:15207:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada44fead80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449615200256:192.168.202.113@tcp:501:ll_cfg_requeue.0 00000100:00100000:1.0:1788572209.618946:0:15207:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada44fead80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449615200384:192.168.202.113@tcp:503:ll_cfg_requeue.0 00000100:00100000:1.0:1788572209.619096:0:15207:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60703d80 going to sleep for 11 seconds 00000100:00100000:1.0:1788572209.629303:0:15207:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada44fead80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449615200384:192.168.202.113@tcp:503:ll_cfg_requeue.0 00000040:00080000:1.0:1788572209.632128:0:86257:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 64768 count 51 00000020:01000000:3.0:1788572209.632829:0:15207:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 52-51 (rc=0) 10000000:01000000:3.0:1788572209.632851:0:15207:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.202.113@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:3.0:1788572209.632896:0:15207:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.202.113@tcp lock: ffff9ada4542b400/0xef30c340bfb45a47 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b831ca81 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572209.632911:0:15207:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.202.113@tcp lock: ffff9ada4542b400/0xef30c340bfb45a47 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b831ca81 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1788572209.632924:0:15207:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:3.0:1788572209.632927:0:15207:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9ada58b0a000 from 7 10000000:01000000:3.0:1788572209.632930:0:15207:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:3.0:1788572209.632985:0:15207:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.202.113@tcp lock: ffff9ada51786000/0xef30c340bfb45a4e lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572209.632995:0:15207:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.202.113@tcp lock: ffff9ada51786000/0xef30c340bfb45a4e lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572209.633014:0:15207:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.202.113@tcp lock: ffff9ada51786000/0xef30c340bfb45a4e lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572209.633048:0:15207:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517a0380 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449615200512:192.168.202.113@tcp:101:ll_cfg_requeue.0 00000100:00100000:3.0:1788572209.633469:0:15207:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada4548f840 going to sleep for 11 seconds 00000100:00100000:3.0:1788572209.642161:0:15207:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517a0380 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449615200512:192.168.202.113@tcp:101:ll_cfg_requeue.0 00010000:00010000:3.0:1788572209.642203:0:15207:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.202.113@tcp lock: ffff9ada51786000/0xef30c340bfb45a4e lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b831ca88 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572209.642217:0:15207:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.202.113@tcp lock: ffff9ada51786000/0xef30c340bfb45a4e lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0xdb7edb52b831ca88 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572209.642314:0:15207:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517a0380 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449615200640:192.168.202.113@tcp:501:ll_cfg_requeue.0 00000100:00100000:3.0:1788572209.642467:0:15207:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada4548f840 going to sleep for 11 seconds 00000100:00100000:3.0:1788572209.645637:0:15207:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517a0380 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449615200640:192.168.202.113@tcp:501:ll_cfg_requeue.0 00000100:00100000:3.0:1788572209.645791:0:15207:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517a0380 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449615200768:192.168.202.113@tcp:503:ll_cfg_requeue.0 00000100:00100000:3.0:1788572209.645972:0:15207:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada4548f840 going to sleep for 11 seconds 00000100:00100000:3.0:1788572209.664521:0:15207:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517a0380 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449615200768:192.168.202.113@tcp:503:ll_cfg_requeue.0 00000040:00080000:2.0:1788572209.665767:0:86258:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x2:0x0] index 64768 count 7 00000020:01000000:1.0:1788572209.666257:0:15207:0:(obd_config.c:2145:class_config_parse_llog()) Processed log params gen 7-6 (rc=0) 10000000:01000000:1.0:1788572209.666282:0:15207:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.202.113@tcp: configuration from log 'params' succeeded (0). 00010000:00010000:1.0:1788572209.666294:0:15207:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.202.113@tcp lock: ffff9ada51786000/0xef30c340bfb45a4e lrc: 3/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b831ca88 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572209.666306:0:15207:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.202.113@tcp lock: ffff9ada51786000/0xef30c340bfb45a4e lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b831ca88 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1788572209.666317:0:15207:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:1.0:1788572209.666320:0:15207:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9ada42e899c0 from 1 10000000:01000000:1.0:1788572209.666323:0:15207:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:1.0:1788572209.666368:0:15207:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.202.113@tcp lock: ffff9ada4542be00/0xef30c340bfb45a55 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572209.666376:0:15207:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.202.113@tcp lock: ffff9ada4542be00/0xef30c340bfb45a55 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572209.666388:0:15207:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.202.113@tcp lock: ffff9ada4542be00/0xef30c340bfb45a55 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572209.666419:0:15207:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada44fead80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449615200896:192.168.202.113@tcp:101:ll_cfg_requeue.0 00000100:00100000:1.0:1788572209.666526:0:15207:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60703d80 going to sleep for 11 seconds 00000100:00100000:1.0:1788572209.690328:0:15207:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada44fead80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449615200896:192.168.202.113@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1788572209.690370:0:15207:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.202.113@tcp lock: ffff9ada4542be00/0xef30c340bfb45a55 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b831ca8f expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572209.690387:0:15207:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.202.113@tcp lock: ffff9ada4542be00/0xef30c340bfb45a55 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0xdb7edb52b831ca8f expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572209.690502:0:15207:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada44fead80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449615201024:192.168.202.113@tcp:501:ll_cfg_requeue.0 00000100:00100000:1.0:1788572209.690600:0:15207:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60703d80 going to sleep for 11 seconds 00000100:00100000:1.0:1788572209.695823:0:15207:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada44fead80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449615201024:192.168.202.113@tcp:501:ll_cfg_requeue.0 10000000:01000000:1.0:1788572209.695920:0:15207:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.202.113@tcp: configuration from log 'lustre-sptlrpc' failed (-2). 00010000:00010000:1.0:1788572209.695934:0:15207:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.202.113@tcp lock: ffff9ada4542be00/0xef30c340bfb45a55 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b831ca8f expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572209.695948:0:15207:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.202.113@tcp lock: ffff9ada4542be00/0xef30c340bfb45a55 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b831ca8f expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572210.045692:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572211.063605:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572212.088838:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572213.114921:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788572213.624238:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level REPLAY/5 force 0 force_next 1 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572213.624247:0:85951:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: REPLAY) 00000100:00100000:0.0:1788572213.624411:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada44fead80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449615201152:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572213.624559:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada44fe9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615201408:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572213.624600:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1788572213.624607:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada44febb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615201280:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572213.624634:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada44fe8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615201536:192.168.202.113@tcp:400:kworker.0 00010000:00010000:0.0:1788572213.630561:0:2384:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 0/0 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1788572213.630585:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada44fe8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615201536:192.168.202.113@tcp:400:kworker.0 00010000:00010000:0.0:1788572213.630608:0:2384:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 0/0 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1788572213.630614:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada44fe9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615201408:192.168.202.113@tcp:400:kworker.0 00010000:00010000:3.0:1788572213.630920:0:33423:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 0 locks of 0 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1788572213.630938:0:33423:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 0 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1788572213.630944:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572213.630945:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572213.630947:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572213.631010:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572213.632415:0:2384:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 0/0 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1788572213.632436:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada44febb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615201280:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572213.632448:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada44fead80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449615201152:192.168.202.113@tcp:400:kworker.0 00010000:00010000:3.0:1788572213.632672:0:15205:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 0 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1788572213.632687:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572213.632689:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572213.632691:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572214.136239:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788572214.292735:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572214.292756:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572214.292759:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572214.292765:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572214.292774:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366801 00000100:00080000:0.0:1788572214.292783:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada516b4700 x1875449615038976/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 512/456 e 0 to 0 dl 1788572168 ref 1 fl New:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572214.292796:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615038464:192.168.202.113@tcp:36:mcreate.0 00000100:00100000:0.0:1788572214.292834:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516b4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615038976:192.168.202.113@tcp:36:rm.0 00000100:00080000:1.0:1788572214.301195:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada516b4700 x1875449615038976/t214748366802(214748366802) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 512/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.301213:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.301217:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.301224:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.301233:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366802 00000100:00080000:1.0:1788572214.301242:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4550a680 x1875449615040640/t214748366803(214748366803) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572179 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.301256:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516b4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615038976:192.168.202.113@tcp:36:rm.0 00000100:00100000:1.0:1788572214.301283:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4550a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615040640:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.313949:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4550a680 x1875449615040640/t214748366803(214748366803) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.313967:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.313970:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.313976:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.313984:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366803 00000100:00080000:1.0:1788572214.313993:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60a36680 x1875449615041024/t214748366804(214748366804) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572179 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.314007:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4550a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615040640:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.314033:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a36680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615041024:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.319668:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60a36680 x1875449615041024/t214748366804(214748366804) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.319690:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.319693:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.319700:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.319709:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366804 00000100:00080000:1.0:1788572214.319719:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60a35880 x1875449615041408/t214748366805(214748366805) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572179 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.319732:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a36680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615041024:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.319770:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615041408:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.331341:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60a35880 x1875449615041408/t214748366805(214748366805) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.331374:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.331378:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.331386:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.331395:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366805 00000100:00080000:1.0:1788572214.331405:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada516b7480 x1875449615041792/t214748366806(214748366806) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572179 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.331420:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a35880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615041408:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.331449:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516b7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615041792:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.341818:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada516b7480 x1875449615041792/t214748366806(214748366806) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.341837:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.341840:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.341847:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.341890:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366806 00000100:00080000:1.0:1788572214.341901:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada516b5880 x1875449615042176/t214748366807(214748366807) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572179 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.341914:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516b7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615041792:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.341939:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516b5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615042176:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.353068:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada516b5880 x1875449615042176/t214748366807(214748366807) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.353940:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.353944:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.353955:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.353966:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366807 00000100:00080000:1.0:1788572214.353977:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60a34000 x1875449615042560/t214748366808(214748366808) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572179 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.353991:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516b5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615042176:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.354021:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a34000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615042560:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.359190:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60a34000 x1875449615042560/t214748366808(214748366808) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.359209:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.359212:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.359220:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.359228:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366808 00000100:00080000:1.0:1788572214.359239:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada516b6d80 x1875449615042944/t214748366809(214748366809) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572179 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.359253:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a34000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615042560:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.359279:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516b6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615042944:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.373338:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada516b6d80 x1875449615042944/t214748366809(214748366809) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.373357:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.373360:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.373368:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.373377:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366809 00000100:00080000:1.0:1788572214.373388:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60a37b80 x1875449615043328/t214748366810(214748366810) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572179 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.373402:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516b6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615042944:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.373427:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615043328:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.380619:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60a37b80 x1875449615043328/t214748366810(214748366810) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.380639:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.380642:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.380649:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.380658:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366810 00000100:00080000:1.0:1788572214.380670:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60a34700 x1875449615043712/t214748366811(214748366811) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572179 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.380685:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615043328:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.380713:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a34700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615043712:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.384660:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60a34700 x1875449615043712/t214748366811(214748366811) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.384679:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.384683:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.384690:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.384699:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366811 00000100:00080000:1.0:1788572214.384711:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada516b5c00 x1875449615044096/t214748366812(214748366812) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572179 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.384725:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a34700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615043712:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.384764:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516b5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615044096:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.393158:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada516b5c00 x1875449615044096/t214748366812(214748366812) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.393177:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.393181:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.393188:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.393195:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366812 00000100:00080000:1.0:1788572214.393206:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45508380 x1875449615044480/t214748366813(214748366813) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572179 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.393220:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516b5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615044096:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.393246:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45508380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615044480:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.399471:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45508380 x1875449615044480/t214748366813(214748366813) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.399490:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.399494:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.399512:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.399520:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366813 00000100:00080000:1.0:1788572214.399533:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada517a2d80 x1875449615044864/t214748366814(214748366814) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572179 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.399545:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45508380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615044480:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.399569:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517a2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615044864:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.411517:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada517a2d80 x1875449615044864/t214748366814(214748366814) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.411536:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.411539:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.411546:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.411554:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366814 00000100:00080000:1.0:1788572214.411566:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60a35500 x1875449615045248/t214748366815(214748366815) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572179 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.411580:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517a2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615044864:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.411605:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615045248:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.417592:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60a35500 x1875449615045248/t214748366815(214748366815) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.417611:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.417614:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.417620:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.417628:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366815 00000100:00080000:1.0:1788572214.417638:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada517a1880 x1875449615045632/t214748366816(214748366816) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572179 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.417651:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a35500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615045248:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.417676:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517a1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615045632:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572214.426582:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada517a1880 x1875449615045632/t214748366816(214748366816) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572214.426603:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572214.426607:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572214.426614:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572214.426623:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366816 00000100:00080000:0.0:1788572214.426646:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada516b7800 x1875449615046016/t214748366817(214748366817) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572179 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572214.426659:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517a1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615045632:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572214.426685:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516b7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615046016:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.439143:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada516b7800 x1875449615046016/t214748366817(214748366817) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.439162:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.439166:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.439174:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.439183:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366817 00000100:00080000:1.0:1788572214.439195:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada516b5f80 x1875449615046400/t214748366818(214748366818) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572179 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.439208:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516b7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615046016:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.439234:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516b5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615046400:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572214.456199:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada516b5f80 x1875449615046400/t214748366818(214748366818) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572214.456217:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572214.456220:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572214.456227:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572214.456236:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366818 00000100:00080000:0.0:1788572214.456248:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada516b4e00 x1875449615046784/t214748366819(214748366819) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572179 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572214.456262:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516b5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615046400:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572214.456289:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516b4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615046784:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572214.461379:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada516b4e00 x1875449615046784/t214748366819(214748366819) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572214.461399:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572214.461403:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572214.461410:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572214.461418:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366819 00000100:00080000:0.0:1788572214.461430:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60a34380 x1875449615047168/t214748366820(214748366820) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572179 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572214.461443:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516b4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615046784:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572214.461468:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a34380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615047168:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572214.468474:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60a34380 x1875449615047168/t214748366820(214748366820) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572214.468493:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572214.468509:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572214.468517:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572214.468526:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366820 00000100:00080000:0.0:1788572214.468540:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45509500 x1875449615047552/t214748366821(214748366821) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572179 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572214.468556:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a34380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615047168:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572214.468587:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45509500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615047552:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572214.474662:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45509500 x1875449615047552/t214748366821(214748366821) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572214.474678:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572214.474681:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572214.474688:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572214.474696:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366821 00000100:00080000:0.0:1788572214.474707:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60a34e00 x1875449615047936/t214748366822(214748366822) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572179 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572214.474717:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45509500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615047552:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572214.474738:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a34e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615047936:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572214.479979:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60a34e00 x1875449615047936/t214748366822(214748366822) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572214.479999:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572214.480002:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572214.480011:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572214.480020:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366822 00000100:00080000:0.0:1788572214.480034:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4550b480 x1875449615048320/t214748366823(214748366823) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572179 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572214.480061:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a34e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615047936:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572214.480088:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4550b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615048320:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.496370:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4550b480 x1875449615048320/t214748366823(214748366823) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.496387:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.496391:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.496397:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.496405:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366823 00000100:00080000:1.0:1788572214.496418:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60a36a00 x1875449615048704/t214748366824(214748366824) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572179 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.496432:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4550b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615048320:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.496457:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a36a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615048704:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.502220:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60a36a00 x1875449615048704/t214748366824(214748366824) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.502247:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.502250:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.502254:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.502260:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366824 00000100:00080000:1.0:1788572214.502271:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4550aa00 x1875449615049088/t214748366825(214748366825) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572179 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.502280:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a36a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615048704:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.502301:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4550aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615049088:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.506027:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4550aa00 x1875449615049088/t214748366825(214748366825) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.506058:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.506062:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.506069:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.506078:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366825 00000100:00080000:1.0:1788572214.506092:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60a35180 x1875449615049472/t214748366826(214748366826) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572179 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.506107:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4550aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615049088:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.506134:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615049472:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.510365:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60a35180 x1875449615049472/t214748366826(214748366826) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.510378:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.510381:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.510385:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.510391:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366826 00000100:00080000:1.0:1788572214.510402:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60a36d80 x1875449615049856/t214748366827(214748366827) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572179 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.510411:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a35180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615049472:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.510430:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a36d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615049856:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.519581:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60a36d80 x1875449615049856/t214748366827(214748366827) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.519597:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.519600:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.519606:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.519612:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366827 00000100:00080000:1.0:1788572214.519626:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60a36300 x1875449615050240/t214748366828(214748366828) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572179 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.519645:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a36d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615049856:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.519668:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a36300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615050240:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.531997:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60a36300 x1875449615050240/t214748366828(214748366828) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.532015:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.532018:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.532025:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.532034:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366828 00000100:00080000:1.0:1788572214.532047:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada516b6a00 x1875449615050624/t214748366829(214748366829) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572179 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.532060:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a36300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615050240:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.532086:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516b6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615050624:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.543357:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada516b6a00 x1875449615050624/t214748366829(214748366829) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.543375:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.543378:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.543385:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.543394:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366829 00000100:00080000:1.0:1788572214.543408:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada516b6680 x1875449615051008/t214748366830(214748366830) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572179 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.543422:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516b6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615050624:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.543444:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516b6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615051008:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.550132:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada516b6680 x1875449615051008/t214748366830(214748366830) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.550149:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.550153:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.550160:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.550170:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366830 00000100:00080000:1.0:1788572214.550184:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45508e00 x1875449615051392/t214748366831(214748366831) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572179 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.550197:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516b6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615051008:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.550222:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45508e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615051392:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.557602:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45508e00 x1875449615051392/t214748366831(214748366831) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.557619:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.557622:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.557629:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.557639:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366831 00000100:00080000:1.0:1788572214.557651:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45508a80 x1875449615051776/t214748366832(214748366832) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572179 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.557663:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45508e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615051392:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.557686:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45508a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615051776:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.568341:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45508a80 x1875449615051776/t214748366832(214748366832) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.568360:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.568363:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.568370:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.568378:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366832 00000100:00080000:1.0:1788572214.568392:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4550b800 x1875449615052160/t214748366833(214748366833) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572179 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.568405:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45508a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615051776:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.568428:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4550b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615052160:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.581307:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4550b800 x1875449615052160/t214748366833(214748366833) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.581327:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.581330:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.581337:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.581347:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366833 00000100:00080000:1.0:1788572214.581361:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45508000 x1875449615052544/t214748366834(214748366834) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572179 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.581374:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4550b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615052160:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.581401:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45508000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615052544:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.588350:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45508000 x1875449615052544/t214748366834(214748366834) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.588368:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.588371:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.588378:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.588387:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366834 00000100:00080000:1.0:1788572214.588403:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45509c00 x1875449615052928/t214748366835(214748366835) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572179 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.588418:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45508000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615052544:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.588450:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45509c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615052928:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.598735:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45509c00 x1875449615052928/t214748366835(214748366835) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.598766:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.598770:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.598779:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.598788:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366835 00000100:00080000:1.0:1788572214.598806:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4550b100 x1875449615053312/t214748366836(214748366836) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572179 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.598821:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45509c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615052928:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.598893:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4550b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615053312:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.606254:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4550b100 x1875449615053312/t214748366836(214748366836) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.606274:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.606277:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.606284:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.606292:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366836 00000100:00080000:1.0:1788572214.606307:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45508700 x1875449615053696/t214748366837(214748366837) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.606320:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4550b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615053312:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.606346:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45508700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615053696:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.618223:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45508700 x1875449615053696/t214748366837(214748366837) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.618243:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.618246:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.618253:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.618262:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366837 00000100:00080000:1.0:1788572214.618288:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45509180 x1875449615054080/t214748366838(214748366838) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.618300:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45508700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615053696:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.618325:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45509180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615054080:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.622845:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45509180 x1875449615054080/t214748366838(214748366838) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.622903:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.622907:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.622914:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.622922:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366838 00000100:00080000:1.0:1788572214.622938:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45509880 x1875449615054464/t214748366839(214748366839) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.622951:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45509180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615054080:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.622974:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45509880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615054464:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.633727:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45509880 x1875449615054464/t214748366839(214748366839) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.634889:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.634895:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.634909:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.634920:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366839 00000100:00080000:1.0:1788572214.634939:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4550ad80 x1875449615054848/t214748366840(214748366840) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.634953:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45509880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615054464:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.634982:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4550ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615054848:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:2.0:1788572214.650288:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449615201664:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572214.650403:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615201920:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572214.650470:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1788572214.650485:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517fc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615201792:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572214.650527:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615202048:192.168.202.113@tcp:400:kworker.0 00000100:00080000:1.0:1788572214.661683:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4550ad80 x1875449615054848/t214748366840(214748366840) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.661702:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.661705:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.661711:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.661718:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366840 00000100:00080000:1.0:1788572214.661736:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4550a300 x1875449615055232/t214748366841(214748366841) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.661762:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4550ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615054848:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.661790:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4550a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615055232:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:3.0:1788572214.662402:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449615201664:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572214.665136:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517fc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615201792:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572214.665161:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517fca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615202048:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572214.665173:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517fea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615201920:192.168.202.113@tcp:400:kworker.0 00000100:00080000:1.0:1788572214.674076:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4550a300 x1875449615055232/t214748366841(214748366841) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.674094:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.674098:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.674105:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.674113:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366841 00000100:00080000:1.0:1788572214.674131:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4550bb80 x1875449615055616/t214748366842(214748366842) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.674144:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4550a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615055232:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.674170:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4550bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615055616:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.681097:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4550bb80 x1875449615055616/t214748366842(214748366842) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.681128:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.681131:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.681138:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.681146:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366842 00000100:00080000:1.0:1788572214.681161:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60a37100 x1875449615056000/t214748366843(214748366843) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.681175:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4550bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615055616:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.681199:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615056000:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.690109:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60a37100 x1875449615056000/t214748366843(214748366843) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.690133:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.690136:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.690143:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.690151:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366843 00000100:00080000:1.0:1788572214.690168:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada516b4a80 x1875449615056384/t214748366844(214748366844) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.690180:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615056000:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.690205:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516b4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615056384:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.695933:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada516b4a80 x1875449615056384/t214748366844(214748366844) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.695949:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.695953:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.695960:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.695967:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366844 00000100:00080000:1.0:1788572214.695982:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada516b4380 x1875449615056768/t214748366845(214748366845) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.695993:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516b4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615056384:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.696017:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516b4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615056768:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.700594:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada516b4380 x1875449615056768/t214748366845(214748366845) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.700609:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.700613:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.700619:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.700628:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366845 00000100:00080000:1.0:1788572214.700645:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada516b4000 x1875449615057152/t214748366846(214748366846) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.700657:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516b4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615056768:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.700679:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516b4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615057152:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.712962:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada516b4000 x1875449615057152/t214748366846(214748366846) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.712977:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.712981:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.712987:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.712994:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366846 00000100:00080000:1.0:1788572214.713009:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada516b6300 x1875449615057536/t214748366847(214748366847) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.713022:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516b4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615057152:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.713046:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516b6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615057536:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.728104:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada516b6300 x1875449615057536/t214748366847(214748366847) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.728177:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.728180:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.728188:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.728196:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366847 00000100:00080000:1.0:1788572214.728214:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada516b5180 x1875449615057920/t214748366848(214748366848) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.728227:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516b6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615057536:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572214.729235:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516b5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615057920:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572214.733195:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada516b5180 x1875449615057920/t214748366848(214748366848) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572214.733213:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572214.733217:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572214.733225:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572214.733234:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366848 00000100:00080000:0.0:1788572214.733252:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada516b7100 x1875449615058304/t214748366849(214748366849) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572214.733266:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516b5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615057920:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572214.733290:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516b7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615058304:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572214.737642:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada516b7100 x1875449615058304/t214748366849(214748366849) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572214.737662:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572214.737666:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572214.737673:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572214.737682:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366849 00000100:00080000:0.0:1788572214.737701:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada516b5500 x1875449615058688/t214748366850(214748366850) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572214.737714:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516b7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615058304:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572214.738067:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516b5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615058688:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572214.742396:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada516b5500 x1875449615058688/t214748366850(214748366850) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572214.742414:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572214.742417:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572214.742424:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572214.742432:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366850 00000100:00080000:0.0:1788572214.742450:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada516b7b80 x1875449615059072/t214748366851(214748366851) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572214.742463:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516b5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615058688:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572214.742485:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada516b7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615059072:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572214.746317:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada516b7b80 x1875449615059072/t214748366851(214748366851) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572214.746336:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572214.746339:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572214.746346:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572214.746354:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366851 00000100:00080000:0.0:1788572214.746375:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60a35c00 x1875449615059456/t214748366852(214748366852) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572214.746388:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada516b7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615059072:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572214.746412:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615059456:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572214.761719:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60a35c00 x1875449615059456/t214748366852(214748366852) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572214.761752:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572214.761755:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572214.761762:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572214.761769:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366852 00000100:00080000:0.0:1788572214.761789:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60a37800 x1875449615059840/t214748366853(214748366853) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572214.762350:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a35c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615059456:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572214.762384:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615059840:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.773602:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60a37800 x1875449615059840/t214748366853(214748366853) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.773619:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.773622:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.773629:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.773638:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366853 00000100:00080000:1.0:1788572214.773657:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45509f80 x1875449615060224/t214748366854(214748366854) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.773670:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615059840:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.773695:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45509f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615060224:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.792508:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45509f80 x1875449615060224/t214748366854(214748366854) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.792530:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.792534:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.792542:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.792550:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366854 00000100:00080000:1.0:1788572214.792572:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455e6300 x1875449615060608/t214748366855(214748366855) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.792587:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45509f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615060224:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.792611:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615060608:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.829908:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455e6300 x1875449615060608/t214748366855(214748366855) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.829928:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.829932:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.829938:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.829947:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366855 00000100:00080000:1.0:1788572214.829967:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada517a1180 x1875449615060992/t214748366856(214748366856) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.829991:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615060608:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.830017:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517a1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615060992:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.835843:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada517a1180 x1875449615060992/t214748366856(214748366856) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.835895:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.835899:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.835906:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.835915:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366856 00000100:00080000:1.0:1788572214.835935:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60a35f80 x1875449615061376/t214748366857(214748366857) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.835948:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517a1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615060992:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.835972:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615061376:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.867908:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60a35f80 x1875449615061376/t214748366857(214748366857) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.867927:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.867930:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.867937:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.867945:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366857 00000100:00080000:1.0:1788572214.867964:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60731c00 x1875449615061760/t214748366858(214748366858) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.867984:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a35f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615061376:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.868010:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60731c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615061760:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.882203:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60731c00 x1875449615061760/t214748366858(214748366858) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.882221:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.882224:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.882231:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.882240:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366858 00000100:00080000:1.0:1788572214.882259:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60730000 x1875449615062144/t214748366859(214748366859) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.882272:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60731c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615061760:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.882300:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60730000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615062144:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.891444:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60730000 x1875449615062144/t214748366859(214748366859) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.891463:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.891466:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.891472:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.891481:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366859 00000100:00080000:1.0:1788572214.891510:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60733b80 x1875449615062528/t214748366860(214748366860) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.891524:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60730000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615062144:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.891550:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60733b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615062528:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.910578:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60733b80 x1875449615062528/t214748366860(214748366860) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.910597:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.910601:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.910609:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.910618:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366860 00000100:00080000:1.0:1788572214.910639:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60732300 x1875449615062912/t214748366861(214748366861) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.910652:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60733b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615062528:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.910677:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60732300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615062912:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.920905:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60732300 x1875449615062912/t214748366861(214748366861) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.920923:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.920926:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.920932:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.920939:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366861 00000100:00080000:1.0:1788572214.920958:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60732d80 x1875449615063296/t214748366862(214748366862) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.920970:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60732300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615062912:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.920993:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60732d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615063296:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.938026:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60732d80 x1875449615063296/t214748366862(214748366862) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.938046:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.938050:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.938057:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.938065:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366862 00000100:00080000:1.0:1788572214.938085:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60733480 x1875449615063680/t214748366863(214748366863) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.938099:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60732d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615063296:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.938135:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60733480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615063680:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.941934:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60733480 x1875449615063680/t214748366863(214748366863) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.941951:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.941955:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.941962:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.941971:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366863 00000100:00080000:1.0:1788572214.941990:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60731880 x1875449615064064/t214748366864(214748366864) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.942002:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60733480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615063680:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.942024:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60731880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615064064:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.948673:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60731880 x1875449615064064/t214748366864(214748366864) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.948692:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.948695:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.948703:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.948711:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366864 00000100:00080000:1.0:1788572214.948735:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60731180 x1875449615064448/t214748366865(214748366865) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.948761:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60731880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615064064:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.948788:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60731180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615064448:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.960273:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60731180 x1875449615064448/t214748366865(214748366865) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.960292:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.960296:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.960303:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.960311:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366865 00000100:00080000:1.0:1788572214.960332:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60730700 x1875449615064832/t214748366866(214748366866) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.960345:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60731180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615064448:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.960372:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60730700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615064832:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.979105:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60730700 x1875449615064832/t214748366866(214748366866) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.979197:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.979201:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.979208:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.979218:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366866 00000100:00080000:1.0:1788572214.979240:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60733100 x1875449615065216/t214748366867(214748366867) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.979254:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60730700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615064832:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.979281:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60733100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615065216:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572214.988607:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60733100 x1875449615065216/t214748366867(214748366867) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.988623:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572214.988627:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572214.988634:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572214.988642:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366867 00000100:00080000:1.0:1788572214.988666:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60731500 x1875449615065600/t214748366868(214748366868) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572214.988679:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60733100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615065216:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572214.988704:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60731500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615065600:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:3.0:1788572215.006764:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60731500 x1875449615065600/t214748366868(214748366868) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572230 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572215.006782:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:3.0:1788572215.006786:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:3.0:1788572215.006793:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572215.006801:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366868 00000100:00080000:3.0:1788572215.006822:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60a4bb80 x1875449615065984/t214748366869(214748366869) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572215.006835:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60731500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615065600:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:3.0:1788572215.006936:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a4bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615065984:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.017236:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60a4bb80 x1875449615065984/t214748366869(214748366869) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.017253:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.017257:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.017264:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.017273:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366869 00000100:00080000:1.0:1788572215.017297:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60730380 x1875449615066368/t214748366870(214748366870) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.017310:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615065984:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.017336:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60730380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615066368:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.033504:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60730380 x1875449615066368/t214748366870(214748366870) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.033523:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.033536:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.033544:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.033553:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366870 00000100:00080000:1.0:1788572215.033582:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60732680 x1875449615066752/t214748366871(214748366871) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.033596:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60730380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615066368:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.033624:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60732680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615066752:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:2.0:1788572215.040295:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60732680 x1875449615066752/t214748366871(214748366871) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788572215.040319:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1788572215.040322:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:2.0:1788572215.040329:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788572215.040337:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366871 00000100:00080000:2.0:1788572215.040359:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60732a00 x1875449615067136/t214748366872(214748366872) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788572215.040371:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60732680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615066752:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:2.0:1788572215.040396:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60732a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615067136:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:3.0:1788572215.049118:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60732a00 x1875449615067136/t214748366872(214748366872) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572215.049138:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:3.0:1788572215.049142:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:3.0:1788572215.049149:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572215.049158:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366872 00000100:00080000:3.0:1788572215.049182:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60a49880 x1875449615067520/t214748366873(214748366873) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572215.049195:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60732a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615067136:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:3.0:1788572215.049221:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a49880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615067520:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:3.0:1788572215.057042:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60a49880 x1875449615067520/t214748366873(214748366873) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572215.057060:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:3.0:1788572215.057063:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:3.0:1788572215.057070:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572215.057078:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366873 00000100:00080000:3.0:1788572215.057103:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60731f80 x1875449615067904/t214748366874(214748366874) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572215.057130:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a49880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615067520:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:3.0:1788572215.057156:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60731f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615067904:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572215.062468:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60731f80 x1875449615067904/t214748366874(214748366874) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.062498:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572215.062502:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572215.062510:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.062518:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366874 00000100:00080000:0.0:1788572215.062552:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60730a80 x1875449615068288/t214748366875(214748366875) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572215.062565:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60731f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615067904:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572215.062590:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60730a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615068288:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572215.077567:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60730a80 x1875449615068288/t214748366875(214748366875) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.077586:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572215.077589:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572215.077596:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.077604:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366875 00000100:00080000:0.0:1788572215.077626:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60733800 x1875449615068672/t214748366876(214748366876) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572215.077640:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60730a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615068288:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572215.077666:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60733800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615068672:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:3.0:1788572215.089918:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60733800 x1875449615068672/t214748366876(214748366876) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572215.089939:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:3.0:1788572215.089943:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:3.0:1788572215.089950:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572215.089959:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366876 00000100:00080000:3.0:1788572215.089984:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60730e00 x1875449615069056/t214748366877(214748366877) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572215.089998:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60733800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615068672:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:3.0:1788572215.090023:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60730e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615069056:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:3.0:1788572215.095245:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60730e00 x1875449615069056/t214748366877(214748366877) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572215.095263:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:3.0:1788572215.095267:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:3.0:1788572215.095273:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572215.095283:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366877 00000100:00080000:3.0:1788572215.095308:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4547d880 x1875449615069440/t214748366878(214748366878) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572215.095321:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60730e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615069056:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:3.0:1788572215.095346:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4547d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615069440:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.112551:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4547d880 x1875449615069440/t214748366878(214748366878) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.112570:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.112574:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.112581:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.112590:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366878 00000100:00080000:1.0:1788572215.112615:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60a49f80 x1875449615069824/t214748366879(214748366879) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.112628:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4547d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615069440:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.112654:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a49f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615069824:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.117850:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60a49f80 x1875449615069824/t214748366879(214748366879) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.118028:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.118032:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.118040:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.118048:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366879 00000100:00080000:1.0:1788572215.118075:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60a49180 x1875449615070208/t214748366880(214748366880) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.118088:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a49f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615069824:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.118204:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a49180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615070208:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.127192:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60a49180 x1875449615070208/t214748366880(214748366880) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.127209:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.127213:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.127220:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.127228:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366880 00000100:00080000:1.0:1788572215.127254:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4547ca80 x1875449615070592/t214748366881(214748366881) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.127268:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a49180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615070208:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.127294:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4547ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615070592:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.132218:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4547ca80 x1875449615070592/t214748366881(214748366881) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.132237:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.132241:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.132248:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.132256:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366881 00000100:00080000:1.0:1788572215.132279:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4547ea00 x1875449615070976/t214748366882(214748366882) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.132293:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4547ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615070592:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.132317:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4547ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615070976:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572215.142198:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4547ea00 x1875449615070976/t214748366882(214748366882) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.142218:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572215.142221:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572215.142228:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.142236:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366882 00000100:00080000:0.0:1788572215.142262:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4547ce00 x1875449615071360/t214748366883(214748366883) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572215.142275:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4547ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615070976:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572215.142299:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4547ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615071360:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572215.146698:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4547ce00 x1875449615071360/t214748366883(214748366883) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.146716:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572215.146719:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572215.146726:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.146746:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366883 00000100:00080000:0.0:1788572215.146770:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60a49c00 x1875449615071744/t214748366884(214748366884) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572215.146784:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4547ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615071360:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572215.146846:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a49c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615071744:192.168.202.113@tcp:36:unlinkmany.0 00010000:00010000:1.0:1788572215.160018:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788572215.163818:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60a49c00 x1875449615071744/t214748366884(214748366884) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.163839:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.163842:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.163849:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.163882:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366884 00000100:00080000:1.0:1788572215.163908:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60a4b480 x1875449615072128/t214748366885(214748366885) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.163921:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a49c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615071744:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.163946:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a4b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615072128:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572215.199538:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60a4b480 x1875449615072128/t214748366885(214748366885) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.199557:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572215.199561:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572215.199569:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.199578:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366885 00000100:00080000:0.0:1788572215.199603:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60a48000 x1875449615072512/t214748366886(214748366886) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572215.199617:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615072128:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572215.199642:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a48000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615072512:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:3.0:1788572215.218360:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60a48000 x1875449615072512/t214748366886(214748366886) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572215.218379:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:3.0:1788572215.218382:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:3.0:1788572215.218389:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572215.218396:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366886 00000100:00080000:3.0:1788572215.218425:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4547f800 x1875449615072896/t214748366887(214748366887) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572215.218439:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a48000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615072512:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:3.0:1788572215.218467:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4547f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615072896:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:3.0:1788572215.228545:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4547f800 x1875449615072896/t214748366887(214748366887) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572215.228564:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:3.0:1788572215.228568:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:3.0:1788572215.228575:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572215.228583:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366887 00000100:00080000:3.0:1788572215.228611:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4547c000 x1875449615073280/t214748366888(214748366888) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572215.228625:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4547f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615072896:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:3.0:1788572215.228650:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4547c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615073280:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:3.0:1788572215.235779:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4547c000 x1875449615073280/t214748366888(214748366888) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572215.235798:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:3.0:1788572215.235801:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:3.0:1788572215.235809:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572215.235817:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366888 00000100:00080000:3.0:1788572215.235844:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4547dc00 x1875449615073664/t214748366889(214748366889) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572215.237274:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4547c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615073280:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:2.0:1788572215.237832:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4547dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615073664:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572215.244849:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4547dc00 x1875449615073664/t214748366889(214748366889) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.245341:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572215.245345:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572215.245354:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.245364:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366889 00000100:00080000:0.0:1788572215.245392:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4547fb80 x1875449615074048/t214748366890(214748366890) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572215.245407:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4547dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615073664:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572215.245435:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4547fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615074048:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572215.249656:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4547fb80 x1875449615074048/t214748366890(214748366890) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.249675:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572215.249678:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572215.249685:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.249693:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366890 00000100:00080000:0.0:1788572215.249719:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4547e300 x1875449615074432/t214748366891(214748366891) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572215.249732:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4547fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615074048:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572215.249769:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4547e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615074432:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572215.254985:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4547e300 x1875449615074432/t214748366891(214748366891) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.255003:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572215.255007:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572215.255013:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.255021:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366891 00000100:00080000:0.0:1788572215.255048:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4547ed80 x1875449615074816/t214748366892(214748366892) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572215.255061:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4547e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615074432:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572215.255085:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4547ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615074816:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572215.259158:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4547ed80 x1875449615074816/t214748366892(214748366892) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.259177:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572215.259180:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572215.259187:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.259195:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366892 00000100:00080000:0.0:1788572215.259223:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4547f480 x1875449615075200/t214748366893(214748366893) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572215.259236:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4547ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615074816:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572215.259260:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4547f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615075200:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572215.267751:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4547f480 x1875449615075200/t214748366893(214748366893) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.267771:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572215.267774:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572215.267782:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.267792:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366893 00000100:00080000:0.0:1788572215.267834:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60a4a300 x1875449615075584/t214748366894(214748366894) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572215.267847:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4547f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615075200:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:2.0:1788572215.268145:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a4a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615075584:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572215.283818:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60a4a300 x1875449615075584/t214748366894(214748366894) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.283837:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572215.283841:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572215.283847:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.283895:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366894 00000100:00080000:0.0:1788572215.283926:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60a4ad80 x1875449615075968/t214748366895(214748366895) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572215.283944:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615075584:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572215.283972:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a4ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615075968:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572215.289558:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60a4ad80 x1875449615075968/t214748366895(214748366895) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.289571:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572215.289574:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572215.289579:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.289585:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366895 00000100:00080000:0.0:1788572215.289610:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4550dc00 x1875449615076352/t214748366896(214748366896) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572215.289619:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a4ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615075968:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572215.289640:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4550dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615076352:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572215.293157:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4550dc00 x1875449615076352/t214748366896(214748366896) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.293177:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572215.293180:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572215.293187:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.293195:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366896 00000100:00080000:0.0:1788572215.293223:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4550df80 x1875449615076736/t214748366897(214748366897) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572215.293237:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4550dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615076352:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572215.293263:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4550df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615076736:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572215.298458:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4550df80 x1875449615076736/t214748366897(214748366897) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.298479:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572215.298493:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572215.298502:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.298510:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366897 00000100:00080000:0.0:1788572215.298548:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4550ce00 x1875449615077120/t214748366898(214748366898) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572215.298561:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4550df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615076736:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572215.298591:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4550ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615077120:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572215.304543:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4550ce00 x1875449615077120/t214748366898(214748366898) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.304561:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572215.304564:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572215.304572:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.304580:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366898 00000100:00080000:0.0:1788572215.304609:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4550e680 x1875449615077504/t214748366899(214748366899) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572215.304624:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4550ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615077120:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572215.304650:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4550e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615077504:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572215.311080:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4550e680 x1875449615077504/t214748366899(214748366899) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.311098:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572215.311101:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572215.311119:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.311126:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366899 00000100:00080000:0.0:1788572215.311163:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4550c380 x1875449615077888/t214748366900(214748366900) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572215.311177:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4550e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615077504:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572215.311204:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4550c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615077888:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572215.317906:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4550c380 x1875449615077888/t214748366900(214748366900) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.317925:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572215.317929:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572215.317936:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.317946:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366900 00000100:00080000:0.0:1788572215.317974:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4550d500 x1875449615078272/t214748366901(214748366901) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572215.317990:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4550c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615077888:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572215.318015:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4550d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615078272:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572215.322746:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4550d500 x1875449615078272/t214748366901(214748366901) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.322764:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572215.322767:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572215.322775:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.322784:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366901 00000100:00080000:0.0:1788572215.322820:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4550f100 x1875449615078656/t214748366902(214748366902) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572215.322834:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4550d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615078272:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572215.322907:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4550f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615078656:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572215.326214:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4550f100 x1875449615078656/t214748366902(214748366902) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.326232:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572215.326235:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572215.326242:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.326250:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366902 00000100:00080000:0.0:1788572215.326282:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4547d180 x1875449615079040/t214748366903(214748366903) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572215.326295:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4550f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615078656:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572215.326325:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4547d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615079040:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572215.331387:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4547d180 x1875449615079040/t214748366903(214748366903) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.331406:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572215.331410:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572215.331416:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.331423:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366903 00000100:00080000:0.0:1788572215.331453:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4547c700 x1875449615079424/t214748366904(214748366904) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572215.331475:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4547d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615079040:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572215.331508:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4547c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615079424:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572215.335374:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4547c700 x1875449615079424/t214748366904(214748366904) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.335394:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572215.335397:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572215.335403:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.335412:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366904 00000100:00080000:0.0:1788572215.335441:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4547f100 x1875449615079808/t214748366905(214748366905) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572215.335455:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4547c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615079424:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572215.335476:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4547f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615079808:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572215.342473:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4547f100 x1875449615079808/t214748366905(214748366905) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.342498:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572215.342500:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572215.342506:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.342513:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366905 00000100:00080000:0.0:1788572215.342543:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4547d500 x1875449615080192/t214748366906(214748366906) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572215.342553:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4547f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615079808:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572215.342578:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4547d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615080192:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572215.346808:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4547d500 x1875449615080192/t214748366906(214748366906) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.346827:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572215.346830:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572215.346838:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.346846:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366906 00000100:00080000:0.0:1788572215.346900:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4550c700 x1875449615080576/t214748366907(214748366907) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572215.346916:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4547d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615080192:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572215.346944:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4550c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615080576:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572215.350746:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4550c700 x1875449615080576/t214748366907(214748366907) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.350768:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572215.350772:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572215.350780:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.350789:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366907 00000100:00080000:0.0:1788572215.350830:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4550d180 x1875449615080960/t214748366908(214748366908) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572180 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572215.350845:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4550c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615080576:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572215.350917:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4550d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615080960:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572215.357984:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4550d180 x1875449615080960/t214748366908(214748366908) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.358006:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572215.358010:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572215.358017:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.358026:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366908 00000100:00080000:0.0:1788572215.358056:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4550d880 x1875449615081344/t214748366909(214748366909) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572215.358071:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4550d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615080960:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572215.358099:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4550d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615081344:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572215.363090:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4550d880 x1875449615081344/t214748366909(214748366909) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.363108:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572215.363112:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572215.363119:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.363127:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366909 00000100:00080000:0.0:1788572215.363166:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4550f480 x1875449615081728/t214748366910(214748366910) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572215.363180:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4550d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615081344:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572215.363205:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4550f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615081728:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572215.368985:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4550f480 x1875449615081728/t214748366910(214748366910) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.369003:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572215.369007:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572215.369014:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.369034:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366910 00000100:00080000:0.0:1788572215.369064:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4550ed80 x1875449615082112/t214748366911(214748366911) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572215.369077:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4550f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615081728:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572215.369098:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4550ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615082112:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572215.375303:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4550ed80 x1875449615082112/t214748366911(214748366911) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.375321:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572215.375333:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572215.375342:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.375349:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366911 00000100:00080000:0.0:1788572215.375381:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4550e300 x1875449615082496/t214748366912(214748366912) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572215.375396:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4550ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615082112:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572215.375422:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4550e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615082496:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572215.382589:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4550e300 x1875449615082496/t214748366912(214748366912) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.382608:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572215.382612:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572215.382619:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.382628:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366912 00000100:00080000:0.0:1788572215.382662:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4550fb80 x1875449615082880/t214748366913(214748366913) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572215.382676:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4550e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615082496:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572215.382715:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4550fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615082880:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572215.387518:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4550fb80 x1875449615082880/t214748366913(214748366913) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.387538:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572215.387541:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572215.387549:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.387558:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366913 00000100:00080000:0.0:1788572215.387597:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4550c000 x1875449615083264/t214748366914(214748366914) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572215.387610:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4550fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615082880:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572215.387635:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4550c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615083264:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572215.391261:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4550c000 x1875449615083264/t214748366914(214748366914) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.391275:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572215.391278:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572215.391283:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.391288:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366914 00000100:00080000:0.0:1788572215.391316:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4547c380 x1875449615083648/t214748366915(214748366915) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572215.391325:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4550c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615083264:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572215.391461:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4547c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615083648:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572215.401896:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4547c380 x1875449615083648/t214748366915(214748366915) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.401916:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572215.401919:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572215.401927:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.401936:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366915 00000100:00080000:0.0:1788572215.401971:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4547e680 x1875449615084032/t214748366916(214748366916) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572215.401984:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4547c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615083648:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572215.402011:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4547e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615084032:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572215.407966:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4547e680 x1875449615084032/t214748366916(214748366916) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.407984:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572215.407988:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572215.407995:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.408004:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366916 00000100:00080000:0.0:1788572215.408037:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4547df80 x1875449615084416/t214748366917(214748366917) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572215.408051:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4547e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615084032:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572215.408076:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4547df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615084416:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572215.422129:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4547df80 x1875449615084416/t214748366917(214748366917) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.422147:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572215.422150:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572215.422157:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572215.422166:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366917 00000100:00080000:0.0:1788572215.422199:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45407b80 x1875449615084800/t214748366918(214748366918) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572215.422214:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4547df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615084416:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572215.422239:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45407b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615084800:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.429458:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45407b80 x1875449615084800/t214748366918(214748366918) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.429475:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.429478:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.429497:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.429505:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366918 00000100:00080000:1.0:1788572215.429541:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4550f800 x1875449615085184/t214748366919(214748366919) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.429555:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45407b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615084800:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.429582:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4550f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615085184:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.436570:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4550f800 x1875449615085184/t214748366919(214748366919) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.436588:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.436592:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.436599:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.436607:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366919 00000100:00080000:1.0:1788572215.436641:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4550ca80 x1875449615085568/t214748366920(214748366920) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.436654:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4550f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615085184:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.436680:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4550ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615085568:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.445788:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4550ca80 x1875449615085568/t214748366920(214748366920) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.445807:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.445810:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.445817:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.445826:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366920 00000100:00080000:1.0:1788572215.445902:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45406300 x1875449615085952/t214748366921(214748366921) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.445916:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4550ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615085568:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.445943:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45406300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615085952:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.454136:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45406300 x1875449615085952/t214748366921(214748366921) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.454155:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.454158:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.454166:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.454174:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366921 00000100:00080000:1.0:1788572215.454207:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45406d80 x1875449615086336/t214748366922(214748366922) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.454220:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45406300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615085952:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.454245:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45406d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615086336:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.458086:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45406d80 x1875449615086336/t214748366922(214748366922) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.458174:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.458178:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.458185:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.458194:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366922 00000100:00080000:1.0:1788572215.458227:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45407480 x1875449615086720/t214748366923(214748366923) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.458241:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45406d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615086336:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.458267:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45407480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615086720:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.470400:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45407480 x1875449615086720/t214748366923(214748366923) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.470420:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.470423:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.470430:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.470439:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366923 00000100:00080000:1.0:1788572215.470474:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4550ea00 x1875449615087104/t214748366924(214748366924) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.470497:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45407480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615086720:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.470524:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4550ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615087104:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.474411:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4550ea00 x1875449615087104/t214748366924(214748366924) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.474429:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.474432:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.474439:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.474448:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366924 00000100:00080000:1.0:1788572215.474479:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada44817480 x1875449615087488/t214748366925(214748366925) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.474502:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4550ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615087104:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.474525:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada44817480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615087488:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.485531:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada44817480 x1875449615087488/t214748366925(214748366925) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.485550:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.485554:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.485562:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.485571:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366925 00000100:00080000:1.0:1788572215.485609:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45405180 x1875449615087872/t214748366926(214748366926) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.485622:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada44817480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615087488:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.485649:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45405180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615087872:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.490666:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45405180 x1875449615087872/t214748366926(214748366926) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.490683:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.490687:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.490694:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.490702:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366926 00000100:00080000:1.0:1788572215.490736:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45404700 x1875449615088256/t214748366927(214748366927) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.490749:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45405180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615087872:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.490776:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45404700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615088256:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.502428:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45404700 x1875449615088256/t214748366927(214748366927) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.502447:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.502451:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.502458:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.502467:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366927 00000100:00080000:1.0:1788572215.502529:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45407100 x1875449615088640/t214748366928(214748366928) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.502548:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45404700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615088256:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.502586:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45407100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615088640:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.524231:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45407100 x1875449615088640/t214748366928(214748366928) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.524256:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.524263:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.524271:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.524283:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366928 00000100:00080000:1.0:1788572215.524328:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45405500 x1875449615089024/t214748366929(214748366929) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.524341:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45407100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615088640:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.524377:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45405500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615089024:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.534590:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45405500 x1875449615089024/t214748366929(214748366929) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.534607:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.534611:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.534618:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.534626:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366929 00000100:00080000:1.0:1788572215.534661:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45404380 x1875449615089408/t214748366930(214748366930) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.534674:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45405500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615089024:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.534701:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45404380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615089408:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.540754:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45404380 x1875449615089408/t214748366930(214748366930) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.540774:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.540777:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.540784:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.540793:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366930 00000100:00080000:1.0:1788572215.540830:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45406680 x1875449615089792/t214748366931(214748366931) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.540844:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45404380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615089408:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.540901:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45406680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615089792:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.546520:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45406680 x1875449615089792/t214748366931(214748366931) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.546539:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.546542:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.546549:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.546557:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366931 00000100:00080000:1.0:1788572215.546592:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45406a00 x1875449615090176/t214748366932(214748366932) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.546606:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45406680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615089792:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.546632:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45406a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615090176:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.555191:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45406a00 x1875449615090176/t214748366932(214748366932) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.555210:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.555213:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.555219:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.555226:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366932 00000100:00080000:1.0:1788572215.555260:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada44814380 x1875449615090560/t214748366933(214748366933) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.555274:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45406a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615090176:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.555301:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada44814380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615090560:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.560358:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada44814380 x1875449615090560/t214748366933(214748366933) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.560378:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.560381:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.560389:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.560397:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366933 00000100:00080000:1.0:1788572215.560433:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada44815180 x1875449615090944/t214748366934(214748366934) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.560445:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada44814380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615090560:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.560470:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada44815180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615090944:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.575586:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada44815180 x1875449615090944/t214748366934(214748366934) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.575605:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.575609:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.575616:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.575625:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366934 00000100:00080000:1.0:1788572215.575660:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada44815500 x1875449615091328/t214748366935(214748366935) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.575673:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada44815180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615090944:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.575696:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada44815500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615091328:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.588987:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada44815500 x1875449615091328/t214748366935(214748366935) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.589006:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.589010:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.589017:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.589026:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366935 00000100:00080000:1.0:1788572215.589063:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45405f80 x1875449615091712/t214748366936(214748366936) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.589078:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada44815500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615091328:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.589114:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45405f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615091712:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.598847:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45405f80 x1875449615091712/t214748366936(214748366936) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.598898:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.598902:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.598909:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.598918:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366936 00000100:00080000:1.0:1788572215.598954:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45404a80 x1875449615092096/t214748366937(214748366937) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.598968:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45405f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615091712:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.598995:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45404a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615092096:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.606798:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45404a80 x1875449615092096/t214748366937(214748366937) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.606818:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.606821:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.606829:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.606838:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366937 00000100:00080000:1.0:1788572215.606909:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45407800 x1875449615092480/t214748366938(214748366938) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.606923:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45404a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615092096:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.606951:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45407800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615092480:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.613532:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45407800 x1875449615092480/t214748366938(214748366938) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.613553:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.613556:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.613704:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.613714:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366938 00000100:00080000:1.0:1788572215.613752:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada44815880 x1875449615092864/t214748366939(214748366939) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.613767:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45407800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615092480:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.613793:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada44815880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615092864:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.634146:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada44815880 x1875449615092864/t214748366939(214748366939) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.634164:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.634168:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.634174:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.634187:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366939 00000100:00080000:1.0:1788572215.634227:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada44816680 x1875449615093248/t214748366940(214748366940) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.634240:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada44815880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615092864:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.634268:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada44816680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615093248:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.638755:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada44816680 x1875449615093248/t214748366940(214748366940) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.638774:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.638778:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.638785:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.638794:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366940 00000100:00080000:1.0:1788572215.638834:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada44817100 x1875449615093632/t214748366941(214748366941) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.638848:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada44816680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615093248:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.638908:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada44817100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615093632:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.644336:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada44817100 x1875449615093632/t214748366941(214748366941) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.644354:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.644357:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.644364:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.644372:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366941 00000100:00080000:1.0:1788572215.644408:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada44814700 x1875449615094016/t214748366942(214748366942) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.644420:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada44817100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615093632:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.644447:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada44814700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615094016:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.650704:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada44814700 x1875449615094016/t214748366942(214748366942) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.650723:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.650726:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.650740:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.650746:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366942 00000100:00080000:1.0:1788572215.650780:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada607b5180 x1875449615094400/t214748366943(214748366943) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.650791:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada44814700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615094016:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.650814:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada607b5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615094400:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.655836:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada607b5180 x1875449615094400/t214748366943(214748366943) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.655882:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.655885:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.655892:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.655900:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366943 00000100:00080000:1.0:1788572215.655937:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada607b4700 x1875449615094784/t214748366944(214748366944) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.655950:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada607b5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615094400:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.655974:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada607b4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615094784:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.663716:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada607b4700 x1875449615094784/t214748366944(214748366944) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.663747:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.663750:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.663759:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.663768:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366944 00000100:00080000:1.0:1788572215.663816:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada607b7100 x1875449615095168/t214748366945(214748366945) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.663830:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada607b4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615094784:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.663889:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada607b7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615095168:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.670310:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada607b7100 x1875449615095168/t214748366945(214748366945) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.670328:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.670332:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.670339:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.670347:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366945 00000100:00080000:1.0:1788572215.670387:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada607b5500 x1875449615095552/t214748366946(214748366946) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.670401:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada607b7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615095168:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.670427:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada607b5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615095552:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.684895:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada607b5500 x1875449615095552/t214748366946(214748366946) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.684913:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.684917:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.684925:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.684934:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366946 00000100:00080000:1.0:1788572215.684972:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45404e00 x1875449615095936/t214748366947(214748366947) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.684985:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada607b5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615095552:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.685009:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45404e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615095936:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.700389:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45404e00 x1875449615095936/t214748366947(214748366947) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.700408:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.700411:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.700419:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.700427:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366947 00000100:00080000:1.0:1788572215.700464:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45405c00 x1875449615096320/t214748366948(214748366948) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.700487:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45404e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615095936:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.700514:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45405c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615096320:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.707182:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45405c00 x1875449615096320/t214748366948(214748366948) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.707199:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.707202:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.707208:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.707217:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366948 00000100:00080000:1.0:1788572215.707252:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45404000 x1875449615096704/t214748366949(214748366949) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.707264:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45405c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615096320:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.707286:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45404000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615096704:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.738271:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45404000 x1875449615096704/t214748366949(214748366949) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.738289:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.738293:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.738300:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.738308:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366949 00000100:00080000:1.0:1788572215.738346:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45405880 x1875449615097088/t214748366950(214748366950) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.738360:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45404000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615096704:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.738386:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45405880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615097088:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.746354:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45405880 x1875449615097088/t214748366950(214748366950) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.746373:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.746376:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.746384:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.746394:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366950 00000100:00080000:1.0:1788572215.746431:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada5165ca80 x1875449615097472/t214748366951(214748366951) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.746447:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45405880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615097088:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.746474:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada5165ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615097472:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.758278:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada5165ca80 x1875449615097472/t214748366951(214748366951) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.758297:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.758300:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.758307:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.758315:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366951 00000100:00080000:1.0:1788572215.758365:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada5165c380 x1875449615097856/t214748366952(214748366952) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.758378:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada5165ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615097472:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.758401:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada5165c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615097856:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.764900:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada5165c380 x1875449615097856/t214748366952(214748366952) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.764920:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.764924:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.764932:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.764941:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366952 00000100:00080000:1.0:1788572215.764980:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada607b6680 x1875449615098240/t214748366953(214748366953) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.764993:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada5165c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615097856:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.765019:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada607b6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615098240:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.781335:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada607b6680 x1875449615098240/t214748366953(214748366953) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.781353:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.781356:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.781364:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.781372:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366953 00000100:00080000:1.0:1788572215.781411:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada607b6a00 x1875449615098624/t214748366954(214748366954) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.781425:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada607b6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615098240:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.781450:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada607b6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615098624:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.791314:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada607b6a00 x1875449615098624/t214748366954(214748366954) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.791331:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.791335:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.791342:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.791351:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366954 00000100:00080000:1.0:1788572215.791388:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada5165e680 x1875449615099008/t214748366955(214748366955) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.791400:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada607b6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615098624:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.791424:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada5165e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615099008:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.811455:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada5165e680 x1875449615099008/t214748366955(214748366955) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.811472:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.811490:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.811498:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.811506:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366955 00000100:00080000:1.0:1788572215.811549:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada5165ea00 x1875449615099392/t214748366956(214748366956) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.811564:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada5165e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615099008:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.811591:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada5165ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615099392:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.822707:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada5165ea00 x1875449615099392/t214748366956(214748366956) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.822740:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.822744:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.822752:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.822763:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366956 00000100:00080000:1.0:1788572215.822806:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada5165ce00 x1875449615099776/t214748366957(214748366957) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.822821:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada5165ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615099392:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.822849:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada5165ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615099776:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.842958:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada5165ce00 x1875449615099776/t214748366957(214748366957) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.842978:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.842982:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.842988:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.842997:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366957 00000100:00080000:1.0:1788572215.843040:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada5165df80 x1875449615100160/t214748366958(214748366958) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.843053:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada5165ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615099776:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.843079:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada5165df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615100160:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.850381:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada5165df80 x1875449615100160/t214748366958(214748366958) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.850399:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.850403:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.850410:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.850418:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366958 00000100:00080000:1.0:1788572215.850458:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada5165f800 x1875449615100544/t214748366959(214748366959) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.850471:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada5165df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615100160:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.850510:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada5165f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615100544:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.855282:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada5165f800 x1875449615100544/t214748366959(214748366959) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.855296:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.855299:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.855304:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.855310:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366959 00000100:00080000:1.0:1788572215.855347:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada5165c000 x1875449615100928/t214748366960(214748366960) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.855356:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada5165f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615100544:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.855377:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada5165c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615100928:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.860880:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada5165c000 x1875449615100928/t214748366960(214748366960) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.860899:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.860902:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.860907:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.860914:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366960 00000100:00080000:1.0:1788572215.860954:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada5165dc00 x1875449615101312/t214748366961(214748366961) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.860963:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada5165c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615100928:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.860984:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada5165dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615101312:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.865069:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada5165dc00 x1875449615101312/t214748366961(214748366961) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.865087:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.865091:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.865166:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.865176:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366961 00000100:00080000:1.0:1788572215.865217:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada5165fb80 x1875449615101696/t214748366962(214748366962) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.865304:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada5165dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615101312:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.865332:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada5165fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615101696:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.871598:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada5165fb80 x1875449615101696/t214748366962(214748366962) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.871617:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.871621:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.871628:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.871636:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366962 00000100:00080000:1.0:1788572215.871678:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada5165e300 x1875449615102080/t214748366963(214748366963) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.871692:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada5165fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615101696:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.871719:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada5165e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615102080:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.880440:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada5165e300 x1875449615102080/t214748366963(214748366963) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.880458:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.880462:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.880469:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.880489:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366963 00000100:00080000:1.0:1788572215.880534:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada607b4e00 x1875449615102464/t214748366964(214748366964) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.880550:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada5165e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615102080:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.880576:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada607b4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615102464:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.891033:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada607b4e00 x1875449615102464/t214748366964(214748366964) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.891051:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.891055:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.891061:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.891070:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366964 00000100:00080000:1.0:1788572215.891909:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada607b5f80 x1875449615102848/t214748366965(214748366965) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.891929:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada607b4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615102464:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.891960:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada607b5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615102848:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.907720:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada607b5f80 x1875449615102848/t214748366965(214748366965) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.907739:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.907742:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.907750:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.907759:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366965 00000100:00080000:1.0:1788572215.907813:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada607b4a80 x1875449615103232/t214748366966(214748366966) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.907827:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada607b5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615102848:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.907852:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada607b4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615103232:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.922344:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada607b4a80 x1875449615103232/t214748366966(214748366966) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.922360:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.922364:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.922371:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.922378:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366966 00000100:00080000:1.0:1788572215.922420:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada5165ed80 x1875449615103616/t214748366967(214748366967) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.922433:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada607b4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615103232:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.922458:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada5165ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615103616:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:2.0:1788572215.937400:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada5165ed80 x1875449615103616/t214748366967(214748366967) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788572215.937415:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1788572215.937419:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:2.0:1788572215.937425:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788572215.937433:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366967 00000100:00080000:2.0:1788572215.937483:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada5165f480 x1875449615104000/t214748366968(214748366968) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788572215.937498:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada5165ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615103616:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:2.0:1788572215.937525:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada5165f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615104000:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.945354:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada5165f480 x1875449615104000/t214748366968(214748366968) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.945371:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.945375:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.945383:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.945391:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366968 00000100:00080000:1.0:1788572215.945436:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada5165d880 x1875449615104384/t214748366969(214748366969) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.945450:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada5165f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615104000:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.945487:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada5165d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615104384:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.955848:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada5165d880 x1875449615104384/t214748366969(214748366969) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.955882:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.955885:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.955892:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.955901:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366969 00000100:00080000:1.0:1788572215.955941:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada607b7800 x1875449615104768/t214748366970(214748366970) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.955954:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada5165d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615104384:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.955981:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada607b7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615104768:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:2.0:1788572215.976219:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada607b7800 x1875449615104768/t214748366970(214748366970) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788572215.976305:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1788572215.976308:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:2.0:1788572215.976317:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788572215.976325:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366970 00000100:00080000:2.0:1788572215.976365:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada5165d180 x1875449615105152/t214748366971(214748366971) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788572215.976378:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada607b7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615104768:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:2.0:1788572215.976403:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada5165d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615105152:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.987620:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada5165d180 x1875449615105152/t214748366971(214748366971) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.987636:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.987640:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.987647:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.987654:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366971 00000100:00080000:1.0:1788572215.987696:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada607b5c00 x1875449615105536/t214748366972(214748366972) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.987719:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada5165d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615105152:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.987753:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada607b5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615105536:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572215.993780:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada607b5c00 x1875449615105536/t214748366972(214748366972) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.993800:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572215.993804:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572215.993811:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572215.993821:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366972 00000100:00080000:1.0:1788572215.993904:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada607b7b80 x1875449615105920/t214748366973(214748366973) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572215.993918:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada607b5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615105536:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572215.993946:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada607b7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615105920:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:2.0:1788572216.006304:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada607b7b80 x1875449615105920/t214748366973(214748366973) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572231 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788572216.006324:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1788572216.006328:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:2.0:1788572216.006336:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788572216.006346:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366973 00000100:00080000:2.0:1788572216.006390:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada607b6300 x1875449615106304/t214748366974(214748366974) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788572216.006403:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada607b7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615105920:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:2.0:1788572216.006428:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada607b6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615106304:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:2.0:1788572216.017319:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada607b6300 x1875449615106304/t214748366974(214748366974) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788572216.017338:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1788572216.017341:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:2.0:1788572216.017348:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788572216.017356:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366974 00000100:00080000:2.0:1788572216.017400:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada607b6d80 x1875449615106688/t214748366975(214748366975) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788572216.017413:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada607b6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615106304:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:2.0:1788572216.017441:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada607b6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615106688:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:3.0:1788572216.033890:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada607b6d80 x1875449615106688/t214748366975(214748366975) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572216.033911:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:3.0:1788572216.033915:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:3.0:1788572216.033923:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572216.033932:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366975 00000100:00080000:3.0:1788572216.033979:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada5165c700 x1875449615107072/t214748366976(214748366976) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572216.033993:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada607b6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615106688:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:3.0:1788572216.034020:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada5165c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615107072:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572216.042436:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada5165c700 x1875449615107072/t214748366976(214748366976) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572216.042455:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572216.042458:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572216.042466:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572216.042486:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366976 00000100:00080000:0.0:1788572216.042532:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada5165f100 x1875449615107456/t214748366977(214748366977) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572216.042546:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada5165c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615107072:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572216.042573:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada5165f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615107456:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.063531:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada5165f100 x1875449615107456/t214748366977(214748366977) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.063552:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.063556:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.063563:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.063572:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366977 00000100:00080000:1.0:1788572216.063621:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada607b5880 x1875449615107840/t214748366978(214748366978) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.063636:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada5165f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615107456:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572216.063848:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada607b5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615107840:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572216.073450:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada607b5880 x1875449615107840/t214748366978(214748366978) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572216.073469:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572216.073483:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572216.073498:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572216.073506:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366978 00000100:00080000:0.0:1788572216.073550:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada607b7480 x1875449615108224/t214748366979(214748366979) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572216.073563:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada607b5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615107840:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572216.073588:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada607b7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615108224:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572216.081845:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada607b7480 x1875449615108224/t214748366979(214748366979) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572216.081957:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572216.081961:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572216.081991:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572216.081999:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366979 00000100:00080000:0.0:1788572216.082081:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada607b4000 x1875449615108608/t214748366980(214748366980) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572216.082129:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada607b7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615108224:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572216.082155:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada607b4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615108608:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572216.088262:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada607b4000 x1875449615108608/t214748366980(214748366980) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572216.088281:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572216.088285:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572216.088292:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572216.088301:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366980 00000100:00080000:0.0:1788572216.088348:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada607b4380 x1875449615108992/t214748366981(214748366981) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572216.088362:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada607b4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615108608:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572216.088388:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada607b4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615108992:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572216.093282:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada607b4380 x1875449615108992/t214748366981(214748366981) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572216.093302:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572216.093305:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572216.093311:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572216.093319:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366981 00000100:00080000:0.0:1788572216.093361:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455e4e00 x1875449615109376/t214748366982(214748366982) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572216.093373:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada607b4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615108992:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572216.093397:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615109376:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572216.100773:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455e4e00 x1875449615109376/t214748366982(214748366982) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572216.100791:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572216.100795:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572216.101214:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572216.101223:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366982 00000100:00080000:0.0:1788572216.101277:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455e5f80 x1875449615109760/t214748366983(214748366983) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572216.101337:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615109376:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572216.101365:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615109760:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572216.109248:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455e5f80 x1875449615109760/t214748366983(214748366983) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572216.109268:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572216.109271:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572216.109279:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572216.109288:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366983 00000100:00080000:0.0:1788572216.109333:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455e4a80 x1875449615110144/t214748366984(214748366984) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572216.109347:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615109760:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572216.109373:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615110144:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572216.112965:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455e4a80 x1875449615110144/t214748366984(214748366984) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572216.112982:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572216.112985:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572216.112992:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572216.113001:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366984 00000100:00080000:0.0:1788572216.113043:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada517a0700 x1875449615110528/t214748366985(214748366985) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572216.113057:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615110144:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572216.113083:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517a0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615110528:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572216.124018:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada517a0700 x1875449615110528/t214748366985(214748366985) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572216.124035:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572216.124039:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572216.124046:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572216.124052:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366985 00000100:00080000:0.0:1788572216.124106:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60568380 x1875449615110912/t214748366986(214748366986) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572216.124119:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517a0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615110528:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572216.124146:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60568380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615110912:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572216.131065:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60568380 x1875449615110912/t214748366986(214748366986) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572216.131084:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572216.131354:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572216.131364:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572216.131372:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366986 00000100:00080000:0.0:1788572216.131417:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada6056a680 x1875449615111296/t214748366987(214748366987) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572216.131432:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60568380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615110912:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572216.131455:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada6056a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615111296:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572216.142098:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada6056a680 x1875449615111296/t214748366987(214748366987) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572216.142115:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572216.142119:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572216.142126:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572216.142134:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366987 00000100:00080000:0.0:1788572216.142179:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60569500 x1875449615111680/t214748366988(214748366988) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572181 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572216.142191:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada6056a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615111296:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572216.142213:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60569500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615111680:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572216.155452:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60569500 x1875449615111680/t214748366988(214748366988) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572216.155479:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572216.155483:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572216.155490:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572216.155499:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366988 00000100:00080000:0.0:1788572216.155545:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada517a3100 x1875449615112064/t214748366989(214748366989) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572216.155557:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60569500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615111680:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572216.155582:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517a3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615112064:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572216.170069:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada517a3100 x1875449615112064/t214748366989(214748366989) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572216.170100:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572216.170103:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572216.170111:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572216.170118:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366989 00000100:00080000:0.0:1788572216.170168:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada517a3480 x1875449615112448/t214748366990(214748366990) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572216.170183:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517a3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615112064:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572216.170210:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615112448:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572216.179085:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada517a3480 x1875449615112448/t214748366990(214748366990) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572216.179102:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572216.179105:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572216.179113:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572216.179121:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366990 00000100:00080000:0.0:1788572216.179167:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada517a2300 x1875449615112832/t214748366991(214748366991) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572216.179181:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615112448:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572216.179205:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517a2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615112832:192.168.202.113@tcp:36:unlinkmany.0 00010000:00010000:0.0:1788572216.183322:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788572216.192923:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada517a2300 x1875449615112832/t214748366991(214748366991) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572216.192939:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572216.192942:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572216.192947:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572216.192958:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366991 00000100:00080000:0.0:1788572216.193000:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada517a1c00 x1875449615113216/t214748366992(214748366992) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572216.193010:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517a2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615112832:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572216.193029:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517a1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615113216:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572216.210161:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada517a1c00 x1875449615113216/t214748366992(214748366992) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572216.210179:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572216.210183:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572216.210191:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572216.210199:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366992 00000100:00080000:0.0:1788572216.210261:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455e5c00 x1875449615113600/t214748366993(214748366993) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572216.210275:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517a1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615113216:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572216.210302:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615113600:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572216.222305:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455e5c00 x1875449615113600/t214748366993(214748366993) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572216.222323:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572216.222327:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572216.222335:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572216.222344:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366993 00000100:00080000:0.0:1788572216.222593:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada6056b100 x1875449615113984/t214748366994(214748366994) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572216.222610:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615113600:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572216.222642:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada6056b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615113984:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.241482:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada6056b100 x1875449615113984/t214748366994(214748366994) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.241503:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.241507:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.241516:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.241524:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366994 00000100:00080000:1.0:1788572216.241575:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60568700 x1875449615114368/t214748366995(214748366995) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.241589:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada6056b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615113984:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.241614:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60568700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615114368:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.258658:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60568700 x1875449615114368/t214748366995(214748366995) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.258676:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.258679:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.258687:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.258696:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366995 00000100:00080000:1.0:1788572216.258755:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60569180 x1875449615114752/t214748366996(214748366996) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.258770:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60568700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615114368:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.258798:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60569180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615114752:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.266672:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60569180 x1875449615114752/t214748366996(214748366996) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.266690:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.266694:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.266700:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.266709:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366996 00000100:00080000:1.0:1788572216.266998:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455e7b80 x1875449615115136/t214748366997(214748366997) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.267014:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60569180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615114752:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.267038:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615115136:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.270801:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455e7b80 x1875449615115136/t214748366997(214748366997) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.270818:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.270822:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.270840:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.270849:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366997 00000100:00080000:1.0:1788572216.270940:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455e6d80 x1875449615115520/t214748366998(214748366998) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.270955:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615115136:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.270978:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615115520:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.275398:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455e6d80 x1875449615115520/t214748366998(214748366998) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.275418:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.275421:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.275428:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.275437:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366998 00000100:00080000:1.0:1788572216.275492:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455e7480 x1875449615115904/t214748366999(214748366999) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.275506:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615115520:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.275530:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615115904:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.280183:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455e7480 x1875449615115904/t214748366999(214748366999) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.280201:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.280204:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.280212:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.280220:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748366999 00000100:00080000:1.0:1788572216.280269:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455e5880 x1875449615116288/t214748367000(214748367000) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.280283:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615115904:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.280309:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615116288:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.284621:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455e5880 x1875449615116288/t214748367000(214748367000) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.284640:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.284643:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.284651:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.284659:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367000 00000100:00080000:1.0:1788572216.284706:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada517a3800 x1875449615116672/t214748367001(214748367001) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.284781:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615116288:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.285082:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517a3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615116672:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.291489:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada517a3800 x1875449615116672/t214748367001(214748367001) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.291507:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.291511:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.291518:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.291526:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367001 00000100:00080000:1.0:1788572216.291573:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60569880 x1875449615117056/t214748367002(214748367002) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.291586:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517a3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615116672:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.291611:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60569880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615117056:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.296685:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60569880 x1875449615117056/t214748367002(214748367002) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.296706:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.296709:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.296726:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.296734:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367002 00000100:00080000:1.0:1788572216.296788:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada6056b480 x1875449615117440/t214748367003(214748367003) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.296813:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60569880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615117056:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.296848:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada6056b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615117440:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.302489:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada6056b480 x1875449615117440/t214748367003(214748367003) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.302507:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.302511:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.302517:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.302526:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367003 00000100:00080000:1.0:1788572216.302579:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada6056ad80 x1875449615117824/t214748367004(214748367004) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.302594:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada6056b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615117440:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.302622:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada6056ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615117824:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.308355:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada6056ad80 x1875449615117824/t214748367004(214748367004) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.308373:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.308376:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.308384:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.308392:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367004 00000100:00080000:1.0:1788572216.308453:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada6056a300 x1875449615118208/t214748367005(214748367005) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.308465:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada6056ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615117824:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.308504:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada6056a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615118208:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.315342:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada6056a300 x1875449615118208/t214748367005(214748367005) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.315359:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.315362:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.315369:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.315377:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367005 00000100:00080000:1.0:1788572216.315422:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada5165d500 x1875449615118592/t214748367006(214748367006) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.315435:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada6056a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615118208:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.315456:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada5165d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615118592:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.320627:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada5165d500 x1875449615118592/t214748367006(214748367006) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.320646:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.320650:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.320657:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.320664:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367006 00000100:00080000:1.0:1788572216.320724:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada454c2300 x1875449615118976/t214748367007(214748367007) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.320737:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada5165d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615118592:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.320762:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615118976:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.325817:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada454c2300 x1875449615118976/t214748367007(214748367007) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.325836:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.325840:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.325847:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.325891:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367007 00000100:00080000:1.0:1788572216.325948:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada454c3b80 x1875449615119360/t214748367008(214748367008) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.325962:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615118976:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.325987:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615119360:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.329904:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada454c3b80 x1875449615119360/t214748367008(214748367008) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.329924:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.329927:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.329934:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.329942:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367008 00000100:00080000:1.0:1788572216.329991:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada454c1c00 x1875449615119744/t214748367009(214748367009) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.330005:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615119360:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.330031:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615119744:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.334008:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada454c1c00 x1875449615119744/t214748367009(214748367009) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.334024:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.334027:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.334034:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.334042:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367009 00000100:00080000:1.0:1788572216.334156:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada454c0000 x1875449615120128/t214748367010(214748367010) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.334170:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615119744:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.334195:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615120128:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.339339:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada454c0000 x1875449615120128/t214748367010(214748367010) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.339358:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.339361:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.339368:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.339376:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367010 00000100:00080000:1.0:1788572216.339424:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada454c3800 x1875449615120512/t214748367011(214748367011) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.339437:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615120128:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.339462:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615120512:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.343357:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada454c3800 x1875449615120512/t214748367011(214748367011) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.343375:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.343378:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.343385:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.343393:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367011 00000100:00080000:1.0:1788572216.343444:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60569c00 x1875449615120896/t214748367012(214748367012) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.343458:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615120512:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.343496:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60569c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615120896:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.347841:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60569c00 x1875449615120896/t214748367012(214748367012) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.347886:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.347889:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.347897:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.347906:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367012 00000100:00080000:1.0:1788572216.347962:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada6056b800 x1875449615121280/t214748367013(214748367013) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.347976:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60569c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615120896:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.347999:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada6056b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615121280:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.352637:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada6056b800 x1875449615121280/t214748367013(214748367013) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.352655:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.352659:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.352665:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.352675:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367013 00000100:00080000:1.0:1788572216.352892:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60568a80 x1875449615121664/t214748367014(214748367014) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.352907:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada6056b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615121280:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.352932:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60568a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615121664:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.357059:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60568a80 x1875449615121664/t214748367014(214748367014) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.357078:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.357081:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.357160:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.357169:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367014 00000100:00080000:1.0:1788572216.357373:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60569f80 x1875449615122048/t214748367015(214748367015) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.357388:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60568a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615121664:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.357414:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60569f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615122048:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.365820:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60569f80 x1875449615122048/t214748367015(214748367015) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.365839:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.365842:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.365849:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.365891:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367015 00000100:00080000:1.0:1788572216.365948:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada454c1f80 x1875449615122432/t214748367016(214748367016) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.365962:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60569f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615122048:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.365987:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615122432:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.369648:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada454c1f80 x1875449615122432/t214748367016(214748367016) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.369662:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.369665:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.369669:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.369675:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367016 00000100:00080000:1.0:1788572216.369779:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada454c0e00 x1875449615122816/t214748367017(214748367017) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.369792:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615122432:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.369911:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615122816:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.373071:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada454c0e00 x1875449615122816/t214748367017(214748367017) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.373134:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.373138:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.373143:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.373148:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367017 00000100:00080000:1.0:1788572216.373196:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada454c2a00 x1875449615123200/t214748367018(214748367018) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.373205:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615122816:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.373224:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615123200:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.376283:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada454c2a00 x1875449615123200/t214748367018(214748367018) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.376301:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.376304:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.376309:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.376315:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367018 00000100:00080000:1.0:1788572216.376365:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada6056aa00 x1875449615123584/t214748367019(214748367019) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.376379:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615123200:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.376406:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada6056aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615123584:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.379511:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada6056aa00 x1875449615123584/t214748367019(214748367019) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.379528:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.379531:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.379538:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.379545:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367019 00000100:00080000:1.0:1788572216.379601:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60568000 x1875449615123968/t214748367020(214748367020) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.379614:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada6056aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615123584:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.379637:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60568000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615123968:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.383381:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60568000 x1875449615123968/t214748367020(214748367020) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.383400:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.383403:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.383410:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.383418:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367020 00000100:00080000:1.0:1788572216.383481:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60568e00 x1875449615124352/t214748367021(214748367021) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.383493:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60568000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615123968:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.383519:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60568e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615124352:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.387020:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60568e00 x1875449615124352/t214748367021(214748367021) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.387035:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.387037:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.387043:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.387049:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367021 00000100:00080000:1.0:1788572216.387108:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada6056bb80 x1875449615124736/t214748367022(214748367022) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.387117:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60568e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615124352:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.387137:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada6056bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615124736:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.390351:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada6056bb80 x1875449615124736/t214748367022(214748367022) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.390366:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.390369:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.390374:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.390380:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367022 00000100:00080000:1.0:1788572216.390427:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada454c0380 x1875449615125120/t214748367023(214748367023) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.390437:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada6056bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615124736:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.390457:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615125120:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.394675:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada454c0380 x1875449615125120/t214748367023(214748367023) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.394694:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.394697:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.394705:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.394713:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367023 00000100:00080000:1.0:1788572216.395003:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada454c1500 x1875449615125504/t214748367024(214748367024) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.395018:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615125120:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.395049:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615125504:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.400414:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada454c1500 x1875449615125504/t214748367024(214748367024) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.400433:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.400436:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.400443:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.400451:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367024 00000100:00080000:1.0:1788572216.400517:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada454c0700 x1875449615125888/t214748367025(214748367025) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.400532:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615125504:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.400559:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615125888:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.405041:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada454c0700 x1875449615125888/t214748367025(214748367025) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.405060:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.405064:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.405071:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.405079:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367025 00000100:00080000:1.0:1788572216.405255:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada454c1180 x1875449615126272/t214748367026(214748367026) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.405269:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615125888:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.405293:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615126272:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.409506:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada454c1180 x1875449615126272/t214748367026(214748367026) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.409526:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.409529:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.409537:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.409546:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367026 00000100:00080000:1.0:1788572216.409602:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada454c1880 x1875449615126656/t214748367027(214748367027) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.409618:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615126272:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.409642:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615126656:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.415080:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada454c1880 x1875449615126656/t214748367027(214748367027) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.415166:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.415170:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.415178:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.415255:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367027 00000100:00080000:1.0:1788572216.415313:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada454cf480 x1875449615127040/t214748367028(214748367028) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.415328:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615126656:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.415354:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454cf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615127040:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.420622:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada454cf480 x1875449615127040/t214748367028(214748367028) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.420642:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.420645:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.420652:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.420661:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367028 00000100:00080000:1.0:1788572216.420727:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada454ce300 x1875449615127424/t214748367029(214748367029) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.420741:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454cf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615127040:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.420766:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454ce300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615127424:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.425570:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada454ce300 x1875449615127424/t214748367029(214748367029) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.425590:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.425593:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.425601:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.425610:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367029 00000100:00080000:1.0:1788572216.425675:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada454ced80 x1875449615127808/t214748367030(214748367030) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.425690:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454ce300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615127424:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.425723:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454ced80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615127808:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.430677:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada454ced80 x1875449615127808/t214748367030(214748367030) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.430696:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.430700:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.430707:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.430725:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367030 00000100:00080000:1.0:1788572216.430779:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada454cd880 x1875449615128192/t214748367031(214748367031) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.430794:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454ced80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615127808:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.430919:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454cd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615128192:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.436575:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada454cd880 x1875449615128192/t214748367031(214748367031) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.436592:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.436594:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.436601:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.436608:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367031 00000100:00080000:1.0:1788572216.436662:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada454cd180 x1875449615128576/t214748367032(214748367032) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.436674:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454cd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615128192:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.436698:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454cd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615128576:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.441582:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada454cd180 x1875449615128576/t214748367032(214748367032) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.441600:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.441603:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.441610:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.441618:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367032 00000100:00080000:1.0:1788572216.441671:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada454cc700 x1875449615128960/t214748367033(214748367033) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.441686:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454cd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615128576:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.441775:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454cc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615128960:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.446558:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada454cc700 x1875449615128960/t214748367033(214748367033) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.446579:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.446582:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.446588:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.446596:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367033 00000100:00080000:1.0:1788572216.446652:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada454c3480 x1875449615129344/t214748367034(214748367034) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.446666:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454cc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615128960:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.446693:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615129344:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.455592:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada454c3480 x1875449615129344/t214748367034(214748367034) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.455611:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.455615:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.455622:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.455631:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367034 00000100:00080000:1.0:1788572216.455685:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada454c2d80 x1875449615129728/t214748367035(214748367035) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.455698:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615129344:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.455736:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615129728:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.459907:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada454c2d80 x1875449615129728/t214748367035(214748367035) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.459928:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.459931:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.459938:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.459945:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367035 00000100:00080000:1.0:1788572216.459997:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada454c3100 x1875449615130112/t214748367036(214748367036) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.460009:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615129728:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.460034:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615130112:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.464693:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada454c3100 x1875449615130112/t214748367036(214748367036) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.464767:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.464771:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.464779:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.464786:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367036 00000100:00080000:1.0:1788572216.464991:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada454c2680 x1875449615130496/t214748367037(214748367037) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.465005:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615130112:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.465029:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615130496:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.468543:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada454c2680 x1875449615130496/t214748367037(214748367037) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.468562:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.468566:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.468573:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.468581:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367037 00000100:00080000:1.0:1788572216.468636:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada454c0a80 x1875449615130880/t214748367038(214748367038) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.468649:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615130496:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.468674:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615130880:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.474280:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada454c0a80 x1875449615130880/t214748367038(214748367038) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.474298:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.474302:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.474309:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.474318:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367038 00000100:00080000:1.0:1788572216.474368:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada603f4700 x1875449615131264/t214748367039(214748367039) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.474380:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454c0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615130880:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.474401:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615131264:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.478375:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada603f4700 x1875449615131264/t214748367039(214748367039) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.478395:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.478399:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.478406:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.478414:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367039 00000100:00080000:1.0:1788572216.478482:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada454cf100 x1875449615131648/t214748367040(214748367040) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.478495:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615131264:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.478519:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454cf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615131648:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.483013:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada454cf100 x1875449615131648/t214748367040(214748367040) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.483032:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.483036:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.483043:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.483051:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367040 00000100:00080000:1.0:1788572216.483177:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada454cc380 x1875449615132032/t214748367041(214748367041) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.483260:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454cf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615131648:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.483287:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454cc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615132032:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.487521:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada454cc380 x1875449615132032/t214748367041(214748367041) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.487540:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.487544:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.487551:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.487559:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367041 00000100:00080000:1.0:1788572216.487614:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada454ce680 x1875449615132416/t214748367042(214748367042) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.487628:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454cc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615132032:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.487653:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454ce680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615132416:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.492056:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada454ce680 x1875449615132416/t214748367042(214748367042) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.492074:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.492077:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.492147:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.492158:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367042 00000100:00080000:1.0:1788572216.492284:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada454cea00 x1875449615132800/t214748367043(214748367043) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.492298:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454ce680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615132416:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.492319:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454cea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615132800:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.496565:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada454cea00 x1875449615132800/t214748367043(214748367043) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.496583:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.496587:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.496594:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.496603:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367043 00000100:00080000:1.0:1788572216.496662:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada454cce00 x1875449615133184/t214748367044(214748367044) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.496677:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454cea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615132800:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.496701:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454cce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615133184:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.501252:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada454cce00 x1875449615133184/t214748367044(214748367044) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.501273:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.501276:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.501285:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.501293:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367044 00000100:00080000:1.0:1788572216.501350:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada454cdf80 x1875449615133568/t214748367045(214748367045) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.501364:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454cce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615133184:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.501390:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454cdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615133568:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.506631:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada454cdf80 x1875449615133568/t214748367045(214748367045) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.506648:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.506651:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.506658:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.506666:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367045 00000100:00080000:1.0:1788572216.506729:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada603f7100 x1875449615133952/t214748367046(214748367046) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.506742:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454cdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615133568:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.506767:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615133952:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.511283:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada603f7100 x1875449615133952/t214748367046(214748367046) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.511301:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.511304:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.511313:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.511321:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367046 00000100:00080000:1.0:1788572216.511383:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada603f5500 x1875449615134336/t214748367047(214748367047) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.511397:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615133952:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.511424:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615134336:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.518331:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada603f5500 x1875449615134336/t214748367047(214748367047) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.518349:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.518353:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.518361:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.518425:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367047 00000100:00080000:1.0:1788572216.518496:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada454cca80 x1875449615134720/t214748367048(214748367048) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.518510:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615134336:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.518536:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454cca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615134720:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.523989:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada454cca80 x1875449615134720/t214748367048(214748367048) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.524010:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.524013:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.524021:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.524029:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367048 00000100:00080000:1.0:1788572216.524379:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada454cf800 x1875449615135104/t214748367049(214748367049) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.524394:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454cca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615134720:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.524420:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454cf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615135104:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.528483:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada454cf800 x1875449615135104/t214748367049(214748367049) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.528501:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.528505:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.528512:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.528520:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367049 00000100:00080000:1.0:1788572216.528580:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada454cc000 x1875449615135488/t214748367050(214748367050) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.528591:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454cf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615135104:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.528614:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454cc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615135488:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.534123:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada454cc000 x1875449615135488/t214748367050(214748367050) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.534141:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.534145:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.534151:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.534159:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367050 00000100:00080000:1.0:1788572216.534221:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada454cdc00 x1875449615135872/t214748367051(214748367051) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.534236:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454cc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615135488:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.534260:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454cdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615135872:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.540014:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada454cdc00 x1875449615135872/t214748367051(214748367051) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.540032:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.540036:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.540043:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.540051:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367051 00000100:00080000:1.0:1788572216.540119:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada454cfb80 x1875449615136256/t214748367052(214748367052) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.540134:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454cdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615135872:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.540160:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454cfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615136256:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.545985:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada454cfb80 x1875449615136256/t214748367052(214748367052) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.546005:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.546008:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.546015:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.546022:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367052 00000100:00080000:1.0:1788572216.546081:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada454cd500 x1875449615136640/t214748367053(214748367053) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.546159:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454cfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615136256:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.546256:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada454cd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615136640:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.551124:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada454cd500 x1875449615136640/t214748367053(214748367053) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.551143:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.551147:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.551155:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.551163:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367053 00000100:00080000:1.0:1788572216.551222:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455e3480 x1875449615137024/t214748367054(214748367054) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.551236:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada454cd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615136640:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.551265:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615137024:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.556011:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455e3480 x1875449615137024/t214748367054(214748367054) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.556029:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.556033:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.556040:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.556048:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367054 00000100:00080000:1.0:1788572216.556169:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455e3b80 x1875449615137408/t214748367055(214748367055) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.556182:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615137024:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.556208:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615137408:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.560657:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455e3b80 x1875449615137408/t214748367055(214748367055) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.560675:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.560679:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.560687:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.560695:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367055 00000100:00080000:1.0:1788572216.561025:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455e1c00 x1875449615137792/t214748367056(214748367056) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.561040:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615137408:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.561063:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615137792:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.565942:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455e1c00 x1875449615137792/t214748367056(214748367056) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.565960:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.565963:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.565969:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.565977:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367056 00000100:00080000:1.0:1788572216.566033:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455e0000 x1875449615138176/t214748367057(214748367057) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.566046:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615137792:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.566068:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615138176:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.570229:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455e0000 x1875449615138176/t214748367057(214748367057) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.570248:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.570252:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.570259:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.570267:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367057 00000100:00080000:1.0:1788572216.570326:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455e3800 x1875449615138560/t214748367058(214748367058) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.570339:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615138176:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.570365:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615138560:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.575680:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455e3800 x1875449615138560/t214748367058(214748367058) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.575960:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.575965:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.575974:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.575983:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367058 00000100:00080000:1.0:1788572216.576042:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455e0a80 x1875449615138944/t214748367059(214748367059) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.576057:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615138560:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.576154:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615138944:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.580304:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455e0a80 x1875449615138944/t214748367059(214748367059) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.580320:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.580324:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.580332:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.580340:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367059 00000100:00080000:1.0:1788572216.580399:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455e1f80 x1875449615139328/t214748367060(214748367060) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.580412:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615138944:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.580438:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615139328:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.584193:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455e1f80 x1875449615139328/t214748367060(214748367060) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.584212:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.584216:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.584222:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.584231:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367060 00000100:00080000:1.0:1788572216.584290:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455e0e00 x1875449615139712/t214748367061(214748367061) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.584303:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615139328:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.584328:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615139712:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.588269:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455e0e00 x1875449615139712/t214748367061(214748367061) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.588288:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.588292:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.588300:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.588308:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367061 00000100:00080000:1.0:1788572216.588367:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada603f4380 x1875449615140096/t214748367062(214748367062) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.588381:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615139712:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.588408:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615140096:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.592347:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada603f4380 x1875449615140096/t214748367062(214748367062) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.592367:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.592370:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.592377:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.592385:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367062 00000100:00080000:1.0:1788572216.592441:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada603f6680 x1875449615140480/t214748367063(214748367063) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.592454:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615140096:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.592487:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615140480:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.596482:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada603f6680 x1875449615140480/t214748367063(214748367063) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.596501:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.596504:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.596512:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.596521:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367063 00000100:00080000:1.0:1788572216.596591:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada603f6a00 x1875449615140864/t214748367064(214748367064) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.596609:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615140480:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.596635:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615140864:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.600918:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada603f6a00 x1875449615140864/t214748367064(214748367064) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.600936:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.600939:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.600945:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.600954:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367064 00000100:00080000:1.0:1788572216.601009:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada603f4e00 x1875449615141248/t214748367065(214748367065) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.601021:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615140864:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.601042:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615141248:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.604920:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada603f4e00 x1875449615141248/t214748367065(214748367065) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.604937:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.604940:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.604947:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.604955:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367065 00000100:00080000:1.0:1788572216.605015:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada603f5f80 x1875449615141632/t214748367066(214748367066) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.605029:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615141248:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.605054:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615141632:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.609146:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada603f5f80 x1875449615141632/t214748367066(214748367066) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.609163:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.609166:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.609173:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.609181:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367066 00000100:00080000:1.0:1788572216.609239:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455e2a00 x1875449615142016/t214748367067(214748367067) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572182 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.609254:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615141632:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.609279:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615142016:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.613375:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455e2a00 x1875449615142016/t214748367067(214748367067) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.613394:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.613397:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.613405:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.613414:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367067 00000100:00080000:1.0:1788572216.613494:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455e2680 x1875449615142400/t214748367068(214748367068) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.613508:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615142016:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.613537:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615142400:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.618432:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455e2680 x1875449615142400/t214748367068(214748367068) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.618452:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.618455:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.618472:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.618480:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367068 00000100:00080000:1.0:1788572216.618540:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455e0380 x1875449615142784/t214748367069(214748367069) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.618553:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615142400:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.618579:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615142784:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.622568:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455e0380 x1875449615142784/t214748367069(214748367069) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.622586:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.622590:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.622597:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.622605:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367069 00000100:00080000:1.0:1788572216.622665:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455e1500 x1875449615143168/t214748367070(214748367070) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.622678:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615142784:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.622703:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615143168:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.627186:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455e1500 x1875449615143168/t214748367070(214748367070) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.627205:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.627208:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.627215:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.627223:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367070 00000100:00080000:1.0:1788572216.627281:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada603f4a80 x1875449615143552/t214748367071(214748367071) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.627296:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615143168:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.627319:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615143552:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.631533:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada603f4a80 x1875449615143552/t214748367071(214748367071) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.631551:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.631554:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.631562:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.631571:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367071 00000100:00080000:1.0:1788572216.631631:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada603f7800 x1875449615143936/t214748367072(214748367072) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.631643:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615143552:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.631669:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615143936:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.636190:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada603f7800 x1875449615143936/t214748367072(214748367072) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.636209:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.636212:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.636219:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.636228:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367072 00000100:00080000:1.0:1788572216.636289:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455e4700 x1875449615144320/t214748367073(214748367073) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.636301:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615143936:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.636324:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615144320:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.640715:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455e4700 x1875449615144320/t214748367073(214748367073) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.640744:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.640748:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.640756:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.640764:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367073 00000100:00080000:1.0:1788572216.641002:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455e0700 x1875449615144704/t214748367074(214748367074) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.641017:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615144320:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.641041:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615144704:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.645460:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455e0700 x1875449615144704/t214748367074(214748367074) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.645489:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.645493:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.645501:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.645510:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367074 00000100:00080000:1.0:1788572216.645573:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455e1180 x1875449615145088/t214748367075(214748367075) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.645587:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615144704:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.645612:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615145088:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.649649:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455e1180 x1875449615145088/t214748367075(214748367075) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.649666:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.649669:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.649676:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.649684:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367075 00000100:00080000:1.0:1788572216.649888:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455e1880 x1875449615145472/t214748367076(214748367076) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.649903:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615145088:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.649929:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615145472:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.653997:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455e1880 x1875449615145472/t214748367076(214748367076) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.654014:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.654017:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.654024:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.654032:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367076 00000100:00080000:1.0:1788572216.654158:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455e2d80 x1875449615145856/t214748367077(214748367077) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.654172:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615145472:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.654198:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615145856:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.658342:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455e2d80 x1875449615145856/t214748367077(214748367077) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.658355:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.658357:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.658362:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.658369:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367077 00000100:00080000:1.0:1788572216.658430:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455e2300 x1875449615146240/t214748367078(214748367078) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.658443:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615145856:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.658477:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615146240:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.662272:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455e2300 x1875449615146240/t214748367078(214748367078) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.662290:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.662293:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.662300:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.662308:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367078 00000100:00080000:1.0:1788572216.662370:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455e3100 x1875449615146624/t214748367079(214748367079) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.662384:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615146240:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.662408:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615146624:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.666953:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455e3100 x1875449615146624/t214748367079(214748367079) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.666970:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.666973:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.666980:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.666988:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367079 00000100:00080000:1.0:1788572216.667056:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60411500 x1875449615147008/t214748367080(214748367080) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.667068:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615146624:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.667105:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60411500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615147008:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.671160:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60411500 x1875449615147008/t214748367080(214748367080) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.671174:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.671177:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.671182:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.671191:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367080 00000100:00080000:1.0:1788572216.671256:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60412680 x1875449615147392/t214748367081(214748367081) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.671270:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60411500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615147008:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.671297:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60412680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615147392:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.676888:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60412680 x1875449615147392/t214748367081(214748367081) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.676903:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.676906:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.676911:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.676917:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367081 00000100:00080000:1.0:1788572216.676977:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60410e00 x1875449615147776/t214748367082(214748367082) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.676987:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60412680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615147392:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.677008:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60410e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615147776:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.680321:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60410e00 x1875449615147776/t214748367082(214748367082) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.680335:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.680337:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.680343:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.680349:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367082 00000100:00080000:1.0:1788572216.680409:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60410380 x1875449615148160/t214748367083(214748367083) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.680422:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60410e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615147776:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.680448:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60410380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615148160:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.690098:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60410380 x1875449615148160/t214748367083(214748367083) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.690119:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.690122:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.690130:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.690138:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367083 00000100:00080000:1.0:1788572216.690204:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60412a00 x1875449615148544/t214748367084(214748367084) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.690217:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60410380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615148160:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.690247:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60412a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615148544:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.704067:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60412a00 x1875449615148544/t214748367084(214748367084) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.704086:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.704100:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.704108:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.704117:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367084 00000100:00080000:1.0:1788572216.704181:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada517a0e00 x1875449615148928/t214748367085(214748367085) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.704195:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60412a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615148544:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.704217:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517a0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615148928:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.710093:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada517a0e00 x1875449615148928/t214748367085(214748367085) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.710111:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.710115:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.710122:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.710130:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367085 00000100:00080000:1.0:1788572216.710199:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada603f5c00 x1875449615149312/t214748367086(214748367086) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.710214:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517a0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615148928:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.710240:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615149312:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.715976:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada603f5c00 x1875449615149312/t214748367086(214748367086) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.715996:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.715999:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.716007:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.716015:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367086 00000100:00080000:1.0:1788572216.716083:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada603f7b80 x1875449615149696/t214748367087(214748367087) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.716108:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615149312:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.716134:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615149696:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.723691:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada603f7b80 x1875449615149696/t214748367087(214748367087) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.723709:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.723712:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.723719:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.723727:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367087 00000100:00080000:1.0:1788572216.723796:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada603f6300 x1875449615150080/t214748367088(214748367088) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.723809:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615149696:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.723835:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615150080:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.729903:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada603f6300 x1875449615150080/t214748367088(214748367088) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.729922:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.729926:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.729934:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.729942:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367088 00000100:00080000:1.0:1788572216.730015:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada603f6d80 x1875449615150464/t214748367089(214748367089) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.730029:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615150080:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.730054:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615150464:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.739548:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada603f6d80 x1875449615150464/t214748367089(214748367089) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.739567:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.739570:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.739577:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.739586:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367089 00000100:00080000:1.0:1788572216.739655:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada603f7480 x1875449615150848/t214748367090(214748367090) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.739668:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615150464:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.739705:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615150848:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.755432:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada603f7480 x1875449615150848/t214748367090(214748367090) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.755449:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.755452:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.755459:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.755477:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367090 00000100:00080000:1.0:1788572216.755544:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada603f5880 x1875449615151232/t214748367091(214748367091) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.755558:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615150848:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.755583:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615151232:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.775217:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada603f5880 x1875449615151232/t214748367091(214748367091) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.775236:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.775239:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.775247:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.775255:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367091 00000100:00080000:1.0:1788572216.775325:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada603f5180 x1875449615151616/t214748367092(214748367092) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.775339:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615151232:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.775365:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615151616:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.780754:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada603f5180 x1875449615151616/t214748367092(214748367092) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.780773:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.780776:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.780783:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.780790:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367092 00000100:00080000:1.0:1788572216.781044:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada603f4000 x1875449615152000/t214748367093(214748367093) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.781059:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615151616:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.781151:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada603f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615152000:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.786973:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada603f4000 x1875449615152000/t214748367093(214748367093) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.786991:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.786994:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.787001:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.787009:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367093 00000100:00080000:1.0:1788572216.787442:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45522680 x1875449615152384/t214748367094(214748367094) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.787471:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada603f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615152000:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.787502:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45522680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615152384:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.793966:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45522680 x1875449615152384/t214748367094(214748367094) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.793985:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.793988:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.793995:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.794004:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367094 00000100:00080000:1.0:1788572216.794090:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45522a00 x1875449615152768/t214748367095(214748367095) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.794107:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45522680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615152384:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.794135:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45522a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615152768:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.813010:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45522a00 x1875449615152768/t214748367095(214748367095) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.813029:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.813032:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.813041:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.813049:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367095 00000100:00080000:1.0:1788572216.813130:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60412300 x1875449615153152/t214748367096(214748367096) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.813144:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45522a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615152768:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.813169:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60412300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615153152:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.819387:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60412300 x1875449615153152/t214748367096(214748367096) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.819409:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.819412:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.819420:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.819428:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367096 00000100:00080000:1.0:1788572216.819516:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60410700 x1875449615153536/t214748367097(214748367097) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.819534:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60412300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615153152:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.819566:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60410700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615153536:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.830051:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60410700 x1875449615153536/t214748367097(214748367097) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.830069:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.830072:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.830092:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.830101:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367097 00000100:00080000:1.0:1788572216.830169:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60411180 x1875449615153920/t214748367098(214748367098) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.830304:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60410700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615153536:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.830333:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60411180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615153920:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.835705:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60411180 x1875449615153920/t214748367098(214748367098) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.835731:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.835735:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.835743:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.835751:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367098 00000100:00080000:1.0:1788572216.835827:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60411880 x1875449615154304/t214748367099(214748367099) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.835841:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60411180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615153920:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.835912:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60411880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615154304:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.841548:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60411880 x1875449615154304/t214748367099(214748367099) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.841568:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.841571:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.841579:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.841587:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367099 00000100:00080000:1.0:1788572216.841653:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60413480 x1875449615154688/t214748367100(214748367100) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.841667:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60411880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615154304:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.841693:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60413480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615154688:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.847045:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60413480 x1875449615154688/t214748367100(214748367100) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.847065:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.847068:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.847136:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.847145:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367100 00000100:00080000:1.0:1788572216.847278:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60412d80 x1875449615155072/t214748367101(214748367101) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.847296:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60413480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615154688:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.847327:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60412d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615155072:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.852704:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60412d80 x1875449615155072/t214748367101(214748367101) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.852729:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.852732:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.852739:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.852749:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367101 00000100:00080000:1.0:1788572216.853004:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60413b80 x1875449615155456/t214748367102(214748367102) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.853019:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60412d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615155072:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.853041:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60413b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615155456:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.857008:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60413b80 x1875449615155456/t214748367102(214748367102) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.857026:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.857029:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.857035:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.857043:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367102 00000100:00080000:1.0:1788572216.857114:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45520e00 x1875449615155840/t214748367103(214748367103) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.857126:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60413b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615155456:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.857147:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45520e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615155840:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.861567:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45520e00 x1875449615155840/t214748367103(214748367103) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.861584:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.861587:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.861594:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.861602:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367103 00000100:00080000:1.0:1788572216.861672:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60411c00 x1875449615156224/t214748367104(214748367104) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.861683:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45520e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615155840:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.861724:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60411c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615156224:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.866515:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60411c00 x1875449615156224/t214748367104(214748367104) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.866532:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.866535:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.866541:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.866550:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367104 00000100:00080000:1.0:1788572216.866617:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60410000 x1875449615156608/t214748367105(214748367105) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.866629:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60411c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615156224:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.866650:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60410000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615156608:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.871622:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60410000 x1875449615156608/t214748367105(214748367105) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.871640:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.871644:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.871651:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.871660:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367105 00000100:00080000:1.0:1788572216.871913:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60413800 x1875449615156992/t214748367106(214748367106) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.871927:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60410000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615156608:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.871950:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60413800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615156992:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.876493:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60413800 x1875449615156992/t214748367106(214748367106) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.876511:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.876514:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.876521:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.876529:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367106 00000100:00080000:1.0:1788572216.876595:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60410a80 x1875449615157376/t214748367107(214748367107) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.876608:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60413800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615156992:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.876632:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60410a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615157376:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.882043:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60410a80 x1875449615157376/t214748367107(214748367107) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.882060:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.882063:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.882070:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.882140:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367107 00000100:00080000:1.0:1788572216.882276:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60411f80 x1875449615157760/t214748367108(214748367108) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.882291:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60410a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615157376:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.882316:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60411f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615157760:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.887414:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60411f80 x1875449615157760/t214748367108(214748367108) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.887435:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.887439:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.887447:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.887455:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367108 00000100:00080000:1.0:1788572216.887539:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60413100 x1875449615158144/t214748367109(214748367109) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.887554:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60411f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615157760:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.887583:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60413100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615158144:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.892272:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60413100 x1875449615158144/t214748367109(214748367109) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.892291:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.892294:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.892301:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.892310:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367109 00000100:00080000:1.0:1788572216.892381:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455a4000 x1875449615158528/t214748367110(214748367110) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.892394:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60413100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615158144:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.892418:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455a4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615158528:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.897720:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455a4000 x1875449615158528/t214748367110(214748367110) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.897742:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.897745:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.897753:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.897761:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367110 00000100:00080000:1.0:1788572216.897836:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455a5c00 x1875449615158912/t214748367111(214748367111) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.897850:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455a4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615158528:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.897922:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615158912:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.902733:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455a5c00 x1875449615158912/t214748367111(214748367111) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.902752:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.902755:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.902762:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.902770:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367111 00000100:00080000:1.0:1788572216.902850:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45521f80 x1875449615159296/t214748367112(214748367112) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.902907:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615158912:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.902930:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45521f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615159296:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.907639:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45521f80 x1875449615159296/t214748367112(214748367112) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.907658:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.907661:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.907667:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.907675:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367112 00000100:00080000:1.0:1788572216.907764:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45520a80 x1875449615159680/t214748367113(214748367113) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.907777:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45521f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615159296:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.907815:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45520a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615159680:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.916384:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45520a80 x1875449615159680/t214748367113(214748367113) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.916402:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.916405:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.916412:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.916421:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367113 00000100:00080000:1.0:1788572216.916499:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45523800 x1875449615160064/t214748367114(214748367114) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.916513:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45520a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615159680:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.916537:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45523800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615160064:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.927119:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45523800 x1875449615160064/t214748367114(214748367114) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.927142:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.927145:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.927151:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.927159:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367114 00000100:00080000:1.0:1788572216.927334:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45520000 x1875449615160448/t214748367115(214748367115) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.927348:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45523800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615160064:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.927374:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45520000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615160448:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.932368:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45520000 x1875449615160448/t214748367115(214748367115) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.932386:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.932390:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.932398:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.932406:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367115 00000100:00080000:1.0:1788572216.932483:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455a7b80 x1875449615160832/t214748367116(214748367116) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.932497:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45520000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615160448:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.932520:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615160832:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.937275:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455a7b80 x1875449615160832/t214748367116(214748367116) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.937293:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.937297:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.937304:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.937311:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367116 00000100:00080000:1.0:1788572216.937383:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455a6300 x1875449615161216/t214748367117(214748367117) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.937396:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615160832:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.937419:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455a6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615161216:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.941663:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455a6300 x1875449615161216/t214748367117(214748367117) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.941683:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.941687:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.941694:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.941703:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367117 00000100:00080000:1.0:1788572216.941792:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455a6d80 x1875449615161600/t214748367118(214748367118) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.941806:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455a6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615161216:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.941831:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615161600:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.946948:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455a6d80 x1875449615161600/t214748367118(214748367118) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.946969:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.946972:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.946980:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.946987:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367118 00000100:00080000:1.0:1788572216.947061:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455a7480 x1875449615161984/t214748367119(214748367119) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.947074:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615161600:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.947110:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615161984:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.951779:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455a7480 x1875449615161984/t214748367119(214748367119) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.951801:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.951804:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.951812:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.951821:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367119 00000100:00080000:1.0:1788572216.951934:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455a5880 x1875449615162880/t214748367120(214748367120) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.951947:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615161984:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.951973:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615162880:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.956823:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455a5880 x1875449615162880/t214748367120(214748367120) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.956840:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.956843:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.956850:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.956888:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367120 00000100:00080000:1.0:1788572216.956965:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455a4380 x1875449615163264/t214748367121(214748367121) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.956978:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615162880:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.957002:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615163264:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.961659:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455a4380 x1875449615163264/t214748367121(214748367121) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.961677:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.961680:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.961686:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.961694:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367121 00000100:00080000:1.0:1788572216.961778:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455a6680 x1875449615163648/t214748367122(214748367122) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.961791:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615163264:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.961815:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615163648:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.965632:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455a6680 x1875449615163648/t214748367122(214748367122) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.965645:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.965648:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.965652:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.965658:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367122 00000100:00080000:1.0:1788572216.965743:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45521c00 x1875449615164032/t214748367123(214748367123) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.965753:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615163648:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.965774:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45521c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615164032:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.969709:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45521c00 x1875449615164032/t214748367123(214748367123) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.969733:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.969736:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.969744:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.969754:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367123 00000100:00080000:1.0:1788572216.969838:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455a4e00 x1875449615164416/t214748367124(214748367124) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.969884:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45521c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615164032:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.969910:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615164416:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.975253:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455a4e00 x1875449615164416/t214748367124(214748367124) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.975272:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.975275:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.975282:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.975290:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367124 00000100:00080000:1.0:1788572216.975366:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455e5180 x1875449615164800/t214748367125(214748367125) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.975381:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615164416:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.975408:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615164800:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.985628:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455e5180 x1875449615164800/t214748367125(214748367125) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.985646:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.985650:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.985658:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.985667:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367125 00000100:00080000:1.0:1788572216.986316:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455e4000 x1875449615165184/t214748367126(214748367126) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.986334:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615164800:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.986358:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615165184:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.992940:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455e4000 x1875449615165184/t214748367126(214748367126) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.992958:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.992961:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.992968:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.992976:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367126 00000100:00080000:1.0:1788572216.993061:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455e7800 x1875449615165568/t214748367127(214748367127) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.993077:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615165184:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.993104:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615165568:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572216.998077:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455e7800 x1875449615165568/t214748367127(214748367127) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.998097:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572216.998100:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572216.998107:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572216.998117:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367127 00000100:00080000:1.0:1788572216.998187:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455e6a00 x1875449615165952/t214748367128(214748367128) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572216.998200:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615165568:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572216.998223:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615165952:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572217.002271:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455e6a00 x1875449615165952/t214748367128(214748367128) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572232 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.002291:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572217.002305:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572217.002313:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.002322:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367128 00000100:00080000:1.0:1788572217.002395:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455e6680 x1875449615166336/t214748367129(214748367129) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572217.002409:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615165952:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.002434:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615166336:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572217.006488:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455e6680 x1875449615166336/t214748367129(214748367129) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.006504:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572217.006508:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572217.006514:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.006522:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367129 00000100:00080000:1.0:1788572217.007027:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455e4380 x1875449615166720/t214748367130(214748367130) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572217.007043:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615166336:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.007068:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615166720:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572217.012024:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455e4380 x1875449615166720/t214748367130(214748367130) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.012043:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572217.012046:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572217.012053:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.012061:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367130 00000100:00080000:1.0:1788572217.012202:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455e5500 x1875449615167104/t214748367131(214748367131) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572217.012217:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615166720:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.012243:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615167104:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572217.017984:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455e5500 x1875449615167104/t214748367131(214748367131) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.018003:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572217.018006:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572217.018014:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.018023:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367131 00000100:00080000:1.0:1788572217.018150:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45523b80 x1875449615167488/t214748367132(214748367132) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572217.018166:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615167104:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.018192:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45523b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615167488:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572217.023693:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45523b80 x1875449615167488/t214748367132(214748367132) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.023895:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572217.023898:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572217.023906:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.023914:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367132 00000100:00080000:1.0:1788572217.023995:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45522300 x1875449615167872/t214748367133(214748367133) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572217.024008:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45523b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615167488:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.024034:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45522300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615167872:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572217.035197:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45522300 x1875449615167872/t214748367133(214748367133) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.035214:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572217.035218:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572217.035225:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.035233:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367133 00000100:00080000:1.0:1788572217.035322:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45522d80 x1875449615168256/t214748367134(214748367134) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572217.035339:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45522300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615167872:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.035367:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45522d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615168256:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572217.042527:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45522d80 x1875449615168256/t214748367134(214748367134) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.042555:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572217.042559:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572217.042567:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.042575:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367134 00000100:00080000:1.0:1788572217.042651:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45523480 x1875449615168640/t214748367135(214748367135) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572217.042665:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45522d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615168256:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.042691:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45523480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615168640:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572217.049975:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45523480 x1875449615168640/t214748367135(214748367135) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.049994:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572217.049997:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572217.050005:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.050025:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367135 00000100:00080000:1.0:1788572217.050107:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45521880 x1875449615169024/t214748367136(214748367136) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572217.050121:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45523480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615168640:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.050147:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45521880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615169024:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572217.055612:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45521880 x1875449615169024/t214748367136(214748367136) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.055630:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572217.055633:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572217.055641:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.055650:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367136 00000100:00080000:1.0:1788572217.055783:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455a5f80 x1875449615169408/t214748367137(214748367137) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572217.055797:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45521880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615169024:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.055831:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615169408:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572217.060642:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455a5f80 x1875449615169408/t214748367137(214748367137) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.060661:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572217.060665:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572217.060673:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.060682:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367137 00000100:00080000:1.0:1788572217.060769:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455a7800 x1875449615169792/t214748367138(214748367138) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572217.060783:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455a5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615169408:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.060808:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615169792:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572217.079843:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455a7800 x1875449615169792/t214748367138(214748367138) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.079884:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572217.079887:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572217.079894:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.079903:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367138 00000100:00080000:1.0:1788572217.079980:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455a4a80 x1875449615170176/t214748367139(214748367139) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572217.079993:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615169792:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.080144:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455a4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615170176:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:3.0:1788572217.091474:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455a4a80 x1875449615170176/t214748367139(214748367139) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572217.091493:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:3.0:1788572217.091496:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:3.0:1788572217.091503:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572217.091510:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367139 00000100:00080000:3.0:1788572217.091586:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455a6a00 x1875449615170560/t214748367140(214748367140) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572217.091598:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455a4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615170176:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:3.0:1788572217.091621:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455a6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615170560:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:3.0:1788572217.103451:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455a6a00 x1875449615170560/t214748367140(214748367140) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572217.103478:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:3.0:1788572217.103482:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:3.0:1788572217.103490:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572217.103497:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367140 00000100:00080000:3.0:1788572217.103575:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455a7100 x1875449615170944/t214748367141(214748367141) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572217.103589:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455a6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615170560:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:3.0:1788572217.103614:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615170944:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:3.0:1788572217.120000:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455a7100 x1875449615170944/t214748367141(214748367141) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572217.120148:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:3.0:1788572217.120152:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:3.0:1788572217.120160:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572217.120169:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367141 00000100:00080000:3.0:1788572217.120247:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455a4700 x1875449615171328/t214748367142(214748367142) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572217.120262:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615170944:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572217.122572:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615171328:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572217.126256:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455a4700 x1875449615171328/t214748367142(214748367142) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.126276:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572217.126280:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572217.126287:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.126295:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367142 00000100:00080000:1.0:1788572217.126376:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455a5500 x1875449615171712/t214748367143(214748367143) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572217.126390:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615171328:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.126415:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615171712:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572217.130788:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455a5500 x1875449615171712/t214748367143(214748367143) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.130808:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572217.130811:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572217.130818:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.130827:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367143 00000100:00080000:1.0:1788572217.130934:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455a5180 x1875449615172096/t214748367144(214748367144) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572217.130947:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615171712:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.130972:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615172096:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572217.134553:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455a5180 x1875449615172096/t214748367144(214748367144) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.134570:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572217.134574:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572217.134581:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.134588:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367144 00000100:00080000:1.0:1788572217.134669:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60530000 x1875449615172480/t214748367145(214748367145) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572217.134683:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615172096:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.134718:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60530000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615172480:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572217.139261:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60530000 x1875449615172480/t214748367145(214748367145) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.139282:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572217.139286:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572217.139293:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.139301:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367145 00000100:00080000:1.0:1788572217.139381:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60533800 x1875449615172864/t214748367146(214748367146) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572217.139394:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60530000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615172480:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.139421:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60533800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615172864:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572217.142597:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60533800 x1875449615172864/t214748367146(214748367146) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.142614:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572217.142617:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572217.142623:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.142631:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367146 00000100:00080000:1.0:1788572217.142778:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60530a80 x1875449615173248/t214748367147(214748367147) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572217.142793:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60533800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615172864:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.142820:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60530a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615173248:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572217.147190:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60530a80 x1875449615173248/t214748367147(214748367147) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.147205:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572217.147208:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572217.147214:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.147222:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367147 00000100:00080000:0.0:1788572217.147300:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60531f80 x1875449615173632/t214748367148(214748367148) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572217.147313:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60530a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615173248:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572217.147340:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60531f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615173632:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572217.151346:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60531f80 x1875449615173632/t214748367148(214748367148) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.151364:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572217.151368:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572217.151375:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.151383:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367148 00000100:00080000:0.0:1788572217.151489:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45521180 x1875449615174016/t214748367149(214748367149) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572217.151504:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60531f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615173632:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572217.151530:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45521180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615174016:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572217.159341:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45521180 x1875449615174016/t214748367149(214748367149) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.159360:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572217.159363:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572217.159370:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.159379:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367149 00000100:00080000:0.0:1788572217.159453:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45520700 x1875449615174400/t214748367150(214748367150) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572217.159478:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45521180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615174016:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572217.159505:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45520700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615174400:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572217.167215:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45520700 x1875449615174400/t214748367150(214748367150) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.167231:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572217.167236:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572217.167243:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.167251:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367150 00000100:00080000:0.0:1788572217.167329:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45523100 x1875449615174784/t214748367151(214748367151) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572217.167342:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45520700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615174400:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572217.167369:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45523100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615174784:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572217.170974:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45523100 x1875449615174784/t214748367151(214748367151) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.170993:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572217.170996:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572217.171011:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.171020:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367151 00000100:00080000:0.0:1788572217.171306:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45521500 x1875449615175168/t214748367152(214748367152) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572217.171321:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45523100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615174784:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572217.171347:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45521500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615175168:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572217.178448:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45521500 x1875449615175168/t214748367152(214748367152) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.178478:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572217.178484:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572217.178490:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.178498:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367152 00000100:00080000:0.0:1788572217.178575:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45520380 x1875449615175552/t214748367153(214748367153) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572183 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572217.178589:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45521500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615175168:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572217.178618:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45520380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615175552:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572217.190267:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45520380 x1875449615175552/t214748367153(214748367153) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.190287:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572217.190290:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572217.190297:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.190311:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367153 00000100:00080000:1.0:1788572217.190395:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45537100 x1875449615175936/t214748367154(214748367154) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572184 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572217.190408:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45520380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615175552:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.190432:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45537100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615175936:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572217.197950:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45537100 x1875449615175936/t214748367154(214748367154) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.197967:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572217.197971:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572217.197979:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.197987:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367154 00000100:00080000:1.0:1788572217.198072:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45537b80 x1875449615176320/t214748367155(214748367155) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572184 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572217.198087:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45537100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615175936:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.198112:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45537b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615176320:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572217.203548:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45537b80 x1875449615176320/t214748367155(214748367155) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.203565:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572217.203568:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572217.203575:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.203584:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367155 00000100:00080000:1.0:1788572217.203663:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45536300 x1875449615176704/t214748367156(214748367156) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572184 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572217.203678:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45537b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615176320:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.203719:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45536300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615176704:192.168.202.113@tcp:36:unlinkmany.0 00010000:00010000:3.0:1788572217.207502:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788572217.208526:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45536300 x1875449615176704/t214748367156(214748367156) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.208553:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572217.208556:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572217.208563:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.208572:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367156 00000100:00080000:1.0:1788572217.208669:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60530e00 x1875449615177088/t214748367157(214748367157) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572184 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572217.208683:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45536300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615176704:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.208712:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60530e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615177088:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572217.214226:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60530e00 x1875449615177088/t214748367157(214748367157) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.214244:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572217.214247:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572217.214252:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.214258:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367157 00000100:00080000:1.0:1788572217.214330:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60532a00 x1875449615177472/t214748367158(214748367158) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572184 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572217.214340:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60530e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615177088:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.214360:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60532a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615177472:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572217.218980:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60532a00 x1875449615177472/t214748367158(214748367158) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.218998:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572217.219001:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572217.219018:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.219026:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367158 00000100:00080000:1.0:1788572217.219113:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60532680 x1875449615177856/t214748367159(214748367159) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572184 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572217.219126:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60532a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615177472:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.219152:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60532680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615177856:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572217.223657:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60532680 x1875449615177856/t214748367159(214748367159) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.223676:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572217.223679:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572217.223685:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.223693:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367159 00000100:00080000:1.0:1788572217.223774:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60530380 x1875449615178240/t214748367160(214748367160) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572184 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572217.223785:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60532680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615177856:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.223821:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60530380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615178240:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572217.228058:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60530380 x1875449615178240/t214748367160(214748367160) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.228077:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572217.228080:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572217.228087:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.228094:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367160 00000100:00080000:1.0:1788572217.228173:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60531500 x1875449615178624/t214748367161(214748367161) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572184 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572217.228188:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60530380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615178240:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.228223:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60531500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615178624:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572217.232530:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60531500 x1875449615178624/t214748367161(214748367161) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.232555:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572217.232559:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572217.232566:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.232574:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367161 00000100:00080000:1.0:1788572217.232654:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60533100 x1875449615179008/t214748367162(214748367162) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572184 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572217.232665:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60531500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615178624:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.232686:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60533100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615179008:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572217.239294:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60533100 x1875449615179008/t214748367162(214748367162) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.239309:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572217.239313:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572217.239318:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.239324:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367162 00000100:00080000:1.0:1788572217.239403:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45536d80 x1875449615179392/t214748367163(214748367163) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572184 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572217.239413:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60533100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615179008:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.239434:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45536d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615179392:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572217.243756:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45536d80 x1875449615179392/t214748367163(214748367163) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.243773:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572217.243776:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572217.243781:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.243787:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367163 00000100:00080000:1.0:1788572217.243883:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60531c00 x1875449615179776/t214748367164(214748367164) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572184 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572217.243897:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45536d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615179392:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.243931:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60531c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615179776:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:3.0:1788572217.248324:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60531c00 x1875449615179776/t214748367164(214748367164) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572217.248343:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:3.0:1788572217.248346:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:3.0:1788572217.248353:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572217.248362:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367164 00000100:00080000:3.0:1788572217.248438:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45537480 x1875449615180160/t214748367165(214748367165) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572184 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572217.248450:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60531c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615179776:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:3.0:1788572217.248482:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45537480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615180160:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:3.0:1788572217.260370:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45537480 x1875449615180160/t214748367165(214748367165) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572217.260395:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:3.0:1788572217.260398:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:3.0:1788572217.260405:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572217.260413:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367165 00000100:00080000:3.0:1788572217.260514:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45535880 x1875449615180544/t214748367166(214748367166) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572184 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572217.260786:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45537480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615180160:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:3.0:1788572217.260814:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45535880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615180544:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572217.268338:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45535880 x1875449615180544/t214748367166(214748367166) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.268360:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572217.268363:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572217.268371:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.268380:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367166 00000100:00080000:0.0:1788572217.268472:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45535180 x1875449615180928/t214748367167(214748367167) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572184 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572217.268488:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45535880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615180544:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572217.268513:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45535180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615180928:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572217.276165:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45535180 x1875449615180928/t214748367167(214748367167) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.276303:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572217.276307:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572217.276315:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.276324:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367167 00000100:00080000:0.0:1788572217.276406:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45534700 x1875449615181312/t214748367168(214748367168) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572184 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572217.276419:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45535180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615180928:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572217.276444:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45534700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615181312:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:3.0:1788572217.281610:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45534700 x1875449615181312/t214748367168(214748367168) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572217.281629:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:3.0:1788572217.281633:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:3.0:1788572217.281640:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572217.281648:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367168 00000100:00080000:3.0:1788572217.281771:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45535500 x1875449615181696/t214748367169(214748367169) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572184 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572217.281794:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45534700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615181312:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:3.0:1788572217.281817:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45535500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615181696:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572217.297884:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45535500 x1875449615181696/t214748367169(214748367169) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.297911:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572217.297914:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572217.297921:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.297927:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367169 00000100:00080000:0.0:1788572217.298004:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60533b80 x1875449615182080/t214748367170(214748367170) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572184 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572217.298018:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45535500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615181696:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572217.298046:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60533b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615182080:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572217.302740:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60533b80 x1875449615182080/t214748367170(214748367170) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.302759:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572217.302763:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572217.302770:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.302777:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367170 00000100:00080000:0.0:1788572217.303097:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60532300 x1875449615182464/t214748367171(214748367171) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572184 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572217.303114:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60533b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615182080:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572217.303142:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60532300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615182464:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572217.309213:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60532300 x1875449615182464/t214748367171(214748367171) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.309228:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572217.309231:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572217.309237:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.309243:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367171 00000100:00080000:1.0:1788572217.309326:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60532d80 x1875449615182848/t214748367172(214748367172) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572184 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572217.309336:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60532300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615182464:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.309356:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60532d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615182848:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572217.314888:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60532d80 x1875449615182848/t214748367172(214748367172) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.314908:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572217.314912:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572217.314919:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.314927:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367172 00000100:00080000:0.0:1788572217.315230:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45534380 x1875449615183232/t214748367173(214748367173) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572184 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572217.315245:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60532d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615182848:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.315313:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45534380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615183232:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572217.319333:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45534380 x1875449615183232/t214748367173(214748367173) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.319346:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572217.319348:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572217.319353:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.319358:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367173 00000100:00080000:1.0:1788572217.319477:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45536680 x1875449615183616/t214748367174(214748367174) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572184 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572217.319488:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45534380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615183232:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.319533:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45536680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615183616:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572217.323754:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45536680 x1875449615183616/t214748367174(214748367174) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.323769:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572217.323772:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572217.323778:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.323784:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367174 00000100:00080000:1.0:1788572217.323974:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45536a00 x1875449615184000/t214748367175(214748367175) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572184 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572217.323985:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45536680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615183616:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.324010:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45536a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615184000:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572217.329289:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45536a00 x1875449615184000/t214748367175(214748367175) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.329309:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572217.329313:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572217.329320:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.329329:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367175 00000100:00080000:1.0:1788572217.329415:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60533480 x1875449615184384/t214748367176(214748367176) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572184 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572217.329428:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45536a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615184000:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.329466:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60533480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615184384:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572217.335640:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60533480 x1875449615184384/t214748367176(214748367176) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.335658:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572217.335661:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572217.335670:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.335678:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367176 00000100:00080000:1.0:1788572217.335761:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60531880 x1875449615184768/t214748367177(214748367177) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572184 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572217.335774:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60533480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615184384:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.335810:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60531880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615184768:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572217.343594:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60531880 x1875449615184768/t214748367177(214748367177) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.343614:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572217.343618:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572217.343625:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.343633:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367177 00000100:00080000:1.0:1788572217.343724:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60531180 x1875449615185152/t214748367178(214748367178) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572184 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572217.343739:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60531880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615184768:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.343764:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60531180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615185152:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572217.350840:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60531180 x1875449615185152/t214748367178(214748367178) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.350908:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572217.350911:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572217.350919:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.350928:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367178 00000100:00080000:1.0:1788572217.351091:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60530700 x1875449615185536/t214748367179(214748367179) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572184 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572217.351124:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60531180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615185152:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.351164:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60530700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615185536:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572217.366727:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60530700 x1875449615185536/t214748367179(214748367179) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.366746:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572217.366750:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572217.366757:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.366765:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367179 00000100:00080000:1.0:1788572217.366845:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60740a80 x1875449615185920/t214748367180(214748367180) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572184 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572217.366874:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60530700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615185536:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.366895:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60740a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615185920:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:3.0:1788572217.374796:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60740a80 x1875449615185920/t214748367180(214748367180) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572217.374816:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:3.0:1788572217.374819:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:3.0:1788572217.374827:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572217.374835:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367180 00000100:00080000:3.0:1788572217.374947:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60740000 x1875449615186304/t214748367181(214748367181) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572184 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572217.374961:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60740a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615185920:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:3.0:1788572217.374989:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60740000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615186304:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572217.382311:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60740000 x1875449615186304/t214748367181(214748367181) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.382330:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572217.382333:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572217.382342:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.382352:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367181 00000100:00080000:0.0:1788572217.382436:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60742a00 x1875449615186688/t214748367182(214748367182) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572184 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572217.382448:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60740000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615186304:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572217.382483:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60742a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615186688:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572217.392302:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60742a00 x1875449615186688/t214748367182(214748367182) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.392320:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572217.392323:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572217.392330:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.392339:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367182 00000100:00080000:0.0:1788572217.392422:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60740e00 x1875449615187072/t214748367183(214748367183) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572184 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572217.392437:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60742a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615186688:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572217.392474:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60740e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615187072:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572217.395724:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60740e00 x1875449615187072/t214748367183(214748367183) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.395744:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572217.395748:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572217.395755:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.395764:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367183 00000100:00080000:0.0:1788572217.395850:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60741f80 x1875449615187456/t214748367184(214748367184) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572184 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572217.395898:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60740e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615187072:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572217.395924:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60741f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615187456:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:2.0:1788572217.408496:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60741f80 x1875449615187456/t214748367184(214748367184) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788572217.408515:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1788572217.408518:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:2.0:1788572217.408525:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788572217.408533:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367184 00000100:00080000:2.0:1788572217.408615:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60743800 x1875449615187840/t214748367185(214748367185) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572184 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788572217.408629:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60741f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615187456:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:2.0:1788572217.408655:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60743800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615187840:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572217.419604:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60743800 x1875449615187840/t214748367185(214748367185) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.419623:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572217.419627:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572217.419634:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.419642:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367185 00000100:00080000:1.0:1788572217.419734:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60741c00 x1875449615188224/t214748367186(214748367186) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572184 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572217.419748:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60743800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615187840:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.419778:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60741c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615188224:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572217.426712:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60741c00 x1875449615188224/t214748367186(214748367186) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.426734:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572217.426737:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572217.426745:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.426754:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367186 00000100:00080000:1.0:1788572217.426851:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60743b80 x1875449615188608/t214748367187(214748367187) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572184 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572217.426905:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60741c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615188224:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.426929:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60743b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615188608:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572217.431099:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60743b80 x1875449615188608/t214748367187(214748367187) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.431118:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572217.431121:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572217.431128:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.431134:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367187 00000100:00080000:1.0:1788572217.431414:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60742300 x1875449615188992/t214748367188(214748367188) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572184 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572217.431426:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60743b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615188608:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.431447:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60742300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615188992:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572217.443784:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60742300 x1875449615188992/t214748367188(214748367188) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.443801:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572217.443804:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572217.443812:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.443820:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367188 00000100:00080000:1.0:1788572217.443941:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60742680 x1875449615189376/t214748367189(214748367189) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572184 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572217.443953:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60742300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615188992:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.443976:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60742680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615189376:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572217.457653:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60742680 x1875449615189376/t214748367189(214748367189) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.457672:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572217.457675:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572217.457682:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.457689:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367189 00000100:00080000:1.0:1788572217.457787:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60740380 x1875449615189760/t214748367190(214748367190) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572184 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572217.457801:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60742680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615189376:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.457828:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60740380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615189760:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:1.0:1788572217.466912:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60740380 x1875449615189760/t214748367190(214748367190) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.466931:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1788572217.466934:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:1.0:1788572217.466941:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572217.466949:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367190 00000100:00080000:1.0:1788572217.467033:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60741500 x1875449615190144/t214748367191(214748367191) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572184 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572217.467047:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60740380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615189760:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:1.0:1788572217.467090:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60741500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615190144:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:2.0:1788572217.483416:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60741500 x1875449615190144/t214748367191(214748367191) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788572217.483431:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1788572217.483434:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:2.0:1788572217.483441:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788572217.483461:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367191 00000100:00080000:2.0:1788572217.483549:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45534e00 x1875449615190528/t214748367192(214748367192) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572184 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788572217.483563:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60741500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615190144:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:2.0:1788572217.483587:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45534e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615190528:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:3.0:1788572217.490577:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45534e00 x1875449615190528/t214748367192(214748367192) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572217.490597:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:3.0:1788572217.490600:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:3.0:1788572217.490607:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572217.490616:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367192 00000100:00080000:3.0:1788572217.490705:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45535f80 x1875449615190912/t214748367193(214748367193) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572185 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572217.490720:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45534e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615190528:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:3.0:1788572217.490747:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45535f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615190912:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572217.502394:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45535f80 x1875449615190912/t214748367193(214748367193) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.502408:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572217.502411:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572217.502418:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.502425:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367193 00000100:00080000:0.0:1788572217.502521:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45534a80 x1875449615191296/t214748367194(214748367194) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572185 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572217.502535:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45535f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615190912:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572217.502560:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45534a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615191296:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:2.0:1788572217.520845:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45534a80 x1875449615191296/t214748367194(214748367194) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788572217.520881:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1788572217.520885:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:2.0:1788572217.520893:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788572217.520902:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367194 00000100:00080000:2.0:1788572217.520989:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45537800 x1875449615191680/t214748367195(214748367195) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572185 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788572217.521002:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45534a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615191296:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:2.0:1788572217.521029:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45537800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615191680:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:2.0:1788572217.533978:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45537800 x1875449615191680/t214748367195(214748367195) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788572217.533992:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1788572217.533995:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:2.0:1788572217.534002:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788572217.534010:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367195 00000100:00080000:2.0:1788572217.534102:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60743100 x1875449615192064/t214748367196(214748367196) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572185 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788572217.534115:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45537800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615191680:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:2.0:1788572217.534140:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60743100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615192064:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:2.0:1788572217.543055:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60743100 x1875449615192064/t214748367196(214748367196) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788572217.543075:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1788572217.543078:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:2.0:1788572217.543085:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788572217.543094:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367196 00000100:00080000:2.0:1788572217.543174:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60740700 x1875449615192448/t214748367197(214748367197) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572185 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788572217.543187:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60743100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615192064:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:2.0:1788572217.543209:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60740700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615192448:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572217.552150:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60740700 x1875449615192448/t214748367197(214748367197) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.552171:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572217.552175:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572217.552182:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.552192:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367197 00000100:00080000:0.0:1788572217.552290:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60741180 x1875449615192832/t214748367198(214748367198) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572185 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572217.552305:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60740700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615192448:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:2.0:1788572217.553143:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60741180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615192832:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572217.565981:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60741180 x1875449615192832/t214748367198(214748367198) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.566491:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572217.566496:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572217.566505:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.566514:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367198 00000100:00080000:0.0:1788572217.566607:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada455e7100 x1875449615193216/t214748367199(214748367199) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572185 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572217.566622:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60741180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615192832:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572217.566647:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455e7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615193216:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572217.571605:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada455e7100 x1875449615193216/t214748367199(214748367199) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.571623:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572217.571627:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572217.571633:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.571647:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367199 00000100:00080000:0.0:1788572217.571751:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada517ff100 x1875449615193600/t214748367200(214748367200) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572185 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572217.571764:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455e7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615193216:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572217.571790:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615193600:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572217.576620:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada517ff100 x1875449615193600/t214748367200(214748367200) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.576638:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572217.576641:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572217.576647:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.576655:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367200 00000100:00080000:0.0:1788572217.576755:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60743480 x1875449615193984/t214748367201(214748367201) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572185 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572217.576770:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615193600:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:0.0:1788572217.576795:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60743480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615193984:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:2.0:1788572217.581552:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60743480 x1875449615193984/t214748367201(214748367201) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788572217.581569:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1788572217.581573:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:2.0:1788572217.581580:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788572217.581961:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367201 00000100:00080000:2.0:1788572217.582120:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada45535c00 x1875449615194368/t214748367202(214748367202) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572185 ref 1 fl New:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788572217.582137:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60743480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615193984:192.168.202.113@tcp:36:unlinkmany.0 00000100:00100000:2.0:1788572217.582165:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45535c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615194368:192.168.202.113@tcp:36:unlinkmany.0 00000100:00080000:0.0:1788572217.589785:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada45535c00 x1875449615194368/t214748367202(214748367202) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572233 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.589804:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:0.0:1788572217.589807:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748366800 gen 9 00000100:00100000:0.0:1788572217.589814:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60a37480 x1875449615038464/t214748366801(214748366801) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572225 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.589824:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 214748366800 last 214748367202 00000100:00080000:0.0:1788572217.589930:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from REPLAY to REPLAY_LOCKS 00000100:00100000:3.0:1788572217.590299:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45535c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615194368:192.168.202.113@tcp:36:unlinkmany.0 00010000:00080000:0.0:1788572217.590343:0:86341:0:(ldlm_request.c:2972:ldlm_lock_replay_thread()) lock replay thread lustre-MDT0000-mdc-ffff9ada58b0a000 to lustre-MDT0000_UUID@192.168.202.113@tcp 00010000:00010000:0.0:1788572217.590348:0:86341:0:(ldlm_request.c:2888:ldlm_cancel_unused_locks_for_replay()) Dropping as many unused locks as possible before replay for namespace lustre-MDT0000-mdc-ffff9ada58b0a000 (400) 00000080:00200000:0.0:1788572217.591320:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x641:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.591535:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467c00/0xef30c340bfb44f34 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x641:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8317d7a expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.591568:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x640:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.591657:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467800/0xef30c340bfb44f26 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x640:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8317d49 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.591680:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x63f:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.591696:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60466e00/0xef30c340bfb44f18 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x63f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8317d18 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.591712:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x63e:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.591725:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467400/0xef30c340bfb44f0a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x63e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8317ce7 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.591745:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x63d:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.591757:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60466200/0xef30c340bfb44efc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x63d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8317cb6 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.591773:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x63c:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.591787:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467200/0xef30c340bfb44eee lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x63c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8317c85 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.591802:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x63b:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.591816:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60466000/0xef30c340bfb44ee0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x63b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8317c54 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.591831:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x63a:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.591844:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779000/0xef30c340bfb44ed2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x63a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8317c23 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.591889:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x639:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.591904:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779200/0xef30c340bfb44ec4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x639:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8317bf2 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.591919:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x638:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.591930:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779800/0xef30c340bfb44eb6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x638:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8317bc1 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.591944:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x637:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.591956:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60778e00/0xef30c340bfb44ea8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x637:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8317b90 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.591978:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x636:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.591989:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779400/0xef30c340bfb44e9a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x636:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8317b5f expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.592080:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x635:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.592096:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779a00/0xef30c340bfb44e8c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x635:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8317b2e expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.592111:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x634:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.592123:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60466600/0xef30c340bfb44e7e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x634:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8317afd expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.592137:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x633:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.592148:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60466c00/0xef30c340bfb44e70 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x633:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8317acc expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.592163:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x632:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.592174:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60778600/0xef30c340bfb44e62 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x632:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8317a9b expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.592189:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x631:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.592200:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60466400/0xef30c340bfb44e54 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x631:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8317a6a expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.592215:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x630:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.592226:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60466800/0xef30c340bfb44e46 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x630:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8317a39 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.592245:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x62f:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.592257:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60466a00/0xef30c340bfb44e38 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x62f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8317a08 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.592271:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x62e:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.592283:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60778200/0xef30c340bfb44e2a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x62e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83179d7 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.592296:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x62d:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.592308:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60778c00/0xef30c340bfb44e1c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x62d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83179a6 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.592322:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x62c:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.592334:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60778800/0xef30c340bfb44e0e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x62c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8317975 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.592348:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x62b:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.592360:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60467600/0xef30c340bfb44e00 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x62b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8317944 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.592373:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x62a:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.592384:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60778400/0xef30c340bfb44df2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x62a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8317913 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.592397:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x629:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.592407:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035d000/0xef30c340bfb44de4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x629:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83178e2 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.592425:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x628:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.592436:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779e00/0xef30c340bfb44dd6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x628:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83178b1 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.592460:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x627:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.592472:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60778a00/0xef30c340bfb44dc8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x627:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8317880 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.592487:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x626:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.592497:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60779600/0xef30c340bfb44dba lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x626:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831784f expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.592512:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x625:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.592522:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4546d600/0xef30c340bfb44dac lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x625:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831781e expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.592536:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x624:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.592546:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60451600/0xef30c340bfb44d9e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x624:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83177ed expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.592562:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x623:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.592572:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60451200/0xef30c340bfb44d90 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x623:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83177bc expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.592656:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x622:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.592670:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51758000/0xef30c340bfb44d82 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x622:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831778b expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.592689:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x621:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.592700:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759000/0xef30c340bfb44d74 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x621:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831775a expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.592714:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x620:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.592725:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51758c00/0xef30c340bfb44d66 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x620:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8317729 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.592737:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x61f:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.592749:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60451c00/0xef30c340bfb44d58 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x61f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83176f8 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.592763:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x61e:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.592774:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60450e00/0xef30c340bfb44d4a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x61e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83176c7 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.592789:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x61d:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.592800:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60451400/0xef30c340bfb44d3c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x61d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8317696 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.592815:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x61c:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.592827:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60451a00/0xef30c340bfb44d2e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x61c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8317665 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.592841:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x61b:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.592873:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60450a00/0xef30c340bfb44d20 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x61b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8317634 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.592894:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x61a:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.592907:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787200/0xef30c340bfb44d12 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x61a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8317603 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.592922:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x619:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.592933:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60450600/0xef30c340bfb44d04 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x619:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83175d2 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.592948:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x618:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.592960:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60450000/0xef30c340bfb44cf6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x618:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83175a1 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.592974:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x617:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.592985:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60450c00/0xef30c340bfb44ce8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x617:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8317570 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.593067:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x616:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.593085:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60450200/0xef30c340bfb44cda lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x616:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831753f expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.593102:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x615:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.593114:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60450400/0xef30c340bfb44ccc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x615:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831750e expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.593128:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x614:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.593140:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51758200/0xef30c340bfb44cbe lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x614:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83174dd expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.593160:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x613:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.593171:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51758400/0xef30c340bfb44cb0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x613:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83174ac expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.593185:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x612:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.593195:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51758800/0xef30c340bfb44ca2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x612:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831747b expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.593208:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x611:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.593218:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759e00/0xef30c340bfb44c94 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x611:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831744a expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.593230:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x610:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.593240:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51758a00/0xef30c340bfb44c86 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x610:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8317419 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.593255:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x60f:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.593267:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759200/0xef30c340bfb44c78 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x60f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83173e8 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.593280:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x60e:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.593291:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759600/0xef30c340bfb44c6a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x60e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83173b7 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.593303:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x60d:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.593314:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60450800/0xef30c340bfb44c5c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x60d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8317386 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.593334:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x60c:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.593345:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60451e00/0xef30c340bfb44c4e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x60c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8317355 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.593360:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x60b:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.593371:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60451000/0xef30c340bfb44c40 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x60b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8317324 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.593385:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x60a:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.593395:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6069b800/0xef30c340bfb44c32 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x60a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83172f3 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.593408:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x609:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.593419:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035ce00/0xef30c340bfb44c24 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x609:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83172c2 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.593434:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x608:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.593445:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035d400/0xef30c340bfb44c16 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x608:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8317291 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.593475:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x607:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.593487:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035c200/0xef30c340bfb44c08 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x607:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8317260 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.593500:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x606:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.593510:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035d800/0xef30c340bfb44bfa lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x606:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831722f expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.593527:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x605:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.593537:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035d600/0xef30c340bfb44bec lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x605:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83171fe expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.593551:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x604:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.593562:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759c00/0xef30c340bfb44bde lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x604:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83171cd expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.593576:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x603:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.593660:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759800/0xef30c340bfb44bd0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x603:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831719c expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.593680:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x602:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.593693:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51758e00/0xef30c340bfb44bc2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x602:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831716b expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.593709:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x601:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.593721:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759400/0xef30c340bfb44bb4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x601:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831713a expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.593736:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x600:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.593746:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51759a00/0xef30c340bfb44ba6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x600:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8317109 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.593759:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5ff:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.593769:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51758600/0xef30c340bfb44b98 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5ff:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83170d8 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.593787:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5fe:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.593798:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4546de00/0xef30c340bfb44b8a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5fe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83170a7 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.593813:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5fd:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.593824:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4546d000/0xef30c340bfb44b7c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5fd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8317076 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.593837:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5fc:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.593847:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4546c000/0xef30c340bfb44b6e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5fc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8317045 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.593886:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5fb:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.593898:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4546dc00/0xef30c340bfb44b60 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5fb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8317014 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.593912:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5fa:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.593924:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4546d800/0xef30c340bfb44b52 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5fa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316fe3 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.593939:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5f9:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.593951:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4546ce00/0xef30c340bfb44b44 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5f9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316fb2 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.593965:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5f8:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.593977:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4546d400/0xef30c340bfb44b36 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5f8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316f81 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.593996:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5f7:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.594081:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035c400/0xef30c340bfb44b28 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5f7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316f50 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.594098:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5f6:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.594110:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51786c00/0xef30c340bfb44b1a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5f6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316f1f expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.594125:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5f5:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.594137:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51786400/0xef30c340bfb44b0c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5f5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316eee expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.594152:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5f4:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.594164:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4546da00/0xef30c340bfb44afe lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5f4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316ebd expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.594179:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5f3:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.594190:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4546d200/0xef30c340bfb44af0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5f3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316e8c expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.594205:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5f2:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.594217:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4546c600/0xef30c340bfb44ae2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5f2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316e5b expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.594232:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5f1:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.594245:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4546c800/0xef30c340bfb44ad4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5f1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316e2a expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.594266:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5f0:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.594278:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45517400/0xef30c340bfb44ac6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5f0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316df9 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.594293:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5ef:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.594304:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17e00/0xef30c340bfb44ab8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5ef:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316dc8 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.594317:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5ee:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.594328:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17c00/0xef30c340bfb44aaa lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5ee:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316d97 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.594341:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5ed:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.594353:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787a00/0xef30c340bfb44a9c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5ed:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316d66 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.594367:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5ec:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.594377:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17800/0xef30c340bfb44a8e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5ec:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316d35 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.594393:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5eb:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.594405:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16e00/0xef30c340bfb44a80 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5eb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316d04 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.594422:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5ea:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.594434:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4546cc00/0xef30c340bfb44a72 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5ea:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316cd3 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.594465:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5e9:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.594477:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4546c200/0xef30c340bfb44a64 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5e9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316ca2 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.594492:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5e8:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.594504:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4546c400/0xef30c340bfb44a56 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5e8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316c71 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.594518:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5e7:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.594529:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17400/0xef30c340bfb44a48 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5e7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316c40 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.594542:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5e6:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.594555:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17a00/0xef30c340bfb44a3a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5e6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316c0f expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.594568:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5e5:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.594579:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16600/0xef30c340bfb44a2c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5e5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316bde expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.594671:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5e4:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.594685:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4546ca00/0xef30c340bfb44a1e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5e4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316bad expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.594700:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5e3:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.594713:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6032b000/0xef30c340bfb44a10 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5e3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316b7c expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.594734:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5e2:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.594746:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6069b200/0xef30c340bfb44a02 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5e2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316b4b expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.594762:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5e1:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.594775:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6069a600/0xef30c340bfb449f4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5e1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316b1a expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.594789:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5e0:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.594801:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6069a200/0xef30c340bfb449e6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5e0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316ae9 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.594817:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5df:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.594829:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16000/0xef30c340bfb449d8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5df:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316ab8 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.594843:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5de:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.594874:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6069a000/0xef30c340bfb449ca lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5de:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316a87 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.594891:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5dd:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.594904:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6069ac00/0xef30c340bfb449bc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5dd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316a56 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.594919:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5dc:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.594931:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6069b600/0xef30c340bfb449ae lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5dc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316a25 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.594950:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5db:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.594962:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6069a400/0xef30c340bfb449a0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5db:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83169f4 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.594977:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5da:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.594989:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16c00/0xef30c340bfb44992 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5da:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83169c3 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.595075:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5d9:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.595092:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16200/0xef30c340bfb44984 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5d9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316992 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.595107:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5d8:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.595119:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16400/0xef30c340bfb44976 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5d8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316961 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.595133:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5d7:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.595146:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6069a800/0xef30c340bfb44968 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5d7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316930 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.595161:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5d6:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.595172:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6069be00/0xef30c340bfb4495a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5d6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83168ff expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.595186:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5d5:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.595196:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6069aa00/0xef30c340bfb4494c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5d5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83168ce expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.595213:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5d4:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.595225:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6069b000/0xef30c340bfb4493e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5d4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831689d expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.595240:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5d3:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.595251:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035d200/0xef30c340bfb44930 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5d3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831686c expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.595266:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5d2:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.595278:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51786600/0xef30c340bfb44922 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5d2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831683b expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.595292:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5d1:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.595305:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035de00/0xef30c340bfb44914 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5d1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831680a expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.595317:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5d0:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.595330:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6069b400/0xef30c340bfb44906 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5d0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83167d9 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.595346:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5cf:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.595358:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16800/0xef30c340bfb448f8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5cf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83167a8 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.595372:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5ce:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.595384:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17600/0xef30c340bfb448ea lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5ce:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316777 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.595403:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5cd:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.595415:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a16a00/0xef30c340bfb448dc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5cd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316746 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.595429:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5cc:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.595442:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a17000/0xef30c340bfb448ce lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5cc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316715 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.595471:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5cb:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.595483:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1e00/0xef30c340bfb448c0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5cb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83166e4 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.595498:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5ca:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.595511:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1400/0xef30c340bfb448b2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5ca:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83166b3 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.595525:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5c9:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.595538:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035c000/0xef30c340bfb448a4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5c9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316682 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.595553:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5c8:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.595565:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035dc00/0xef30c340bfb44896 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5c8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316651 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.595580:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5c7:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.595671:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035da00/0xef30c340bfb44888 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5c7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316620 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.595694:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5c6:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.595708:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6069bc00/0xef30c340bfb4487a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5c6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83165ef expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.595724:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5c5:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.595736:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6069ae00/0xef30c340bfb4486c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5c5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83165be expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.595753:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5c4:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.595764:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51786200/0xef30c340bfb4485e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5c4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831658d expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.595777:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5c3:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.595789:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787000/0xef30c340bfb44850 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5c3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831655c expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.595802:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5c2:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.595813:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45519800/0xef30c340bfb44842 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5c2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831652b expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.595829:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5c1:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.595842:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5e00/0xef30c340bfb44834 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5c1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83164fa expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.595887:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5c0:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.595900:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5800/0xef30c340bfb44826 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5c0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83164c9 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.595919:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5bf:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.595932:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1200/0xef30c340bfb44818 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5bf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316498 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.595946:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5be:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.595957:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0000/0xef30c340bfb4480a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5be:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316467 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.595972:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5bd:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.595983:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0600/0xef30c340bfb447fc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5bd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316436 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.595995:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5bc:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.596082:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0c00/0xef30c340bfb447ee lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5bc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316405 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.596099:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5bb:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.596110:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0200/0xef30c340bfb447e0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5bb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83163d4 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.596124:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5ba:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.596136:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0400/0xef30c340bfb447d2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5ba:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83163a3 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.596153:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5b9:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.596167:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6069ba00/0xef30c340bfb447c4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5b9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316372 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.596187:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5b8:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.596197:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60315a00/0xef30c340bfb447b6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5b8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316341 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.596210:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5b7:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.596220:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60314600/0xef30c340bfb447a8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5b7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316310 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.596233:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5b6:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.596244:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606cbe00/0xef30c340bfb4479a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5b6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83162df expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.596258:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5b5:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.596271:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606ca600/0xef30c340bfb4478c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5b5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83162ae expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.596284:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5b4:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.596295:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606ca200/0xef30c340bfb4477e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5b4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831627d expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.596308:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5b3:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.596319:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606cb000/0xef30c340bfb44770 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5b3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831624c expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.596331:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5b2:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.596342:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606cb600/0xef30c340bfb44762 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5b2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831621b expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.596360:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5b1:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.596371:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60314200/0xef30c340bfb44754 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5b1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83161ea expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.596385:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5b0:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.596396:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60315000/0xef30c340bfb44746 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5b0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83161b9 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.596410:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5af:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.596421:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60315c00/0xef30c340bfb44738 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5af:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316188 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.596434:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5ae:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.596444:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606cbc00/0xef30c340bfb4472a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5ae:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316157 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.596470:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5ad:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.596483:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606cb800/0xef30c340bfb4471c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5ad:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316126 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.596497:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5ac:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.596507:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606cae00/0xef30c340bfb4470e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5ac:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83160f5 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.596520:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5ab:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.596529:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606cb400/0xef30c340bfb44700 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5ab:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83160c4 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.596548:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5aa:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.596560:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606cba00/0xef30c340bfb446f2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5aa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316093 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.596572:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5a9:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.596649:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606cb200/0xef30c340bfb446e4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5a9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316062 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.596669:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5a8:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.596681:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0800/0xef30c340bfb446d6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5a8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316031 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.596694:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5a7:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.596705:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1000/0xef30c340bfb446c8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5a7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8316000 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.596719:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5a6:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.596731:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0a00/0xef30c340bfb446ba lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5a6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315fcf expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.596745:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5a5:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.596758:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1600/0xef30c340bfb446ac lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5a5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315f9e expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.596773:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5a4:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.596786:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606ca000/0xef30c340bfb4469e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5a4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315f6d expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.596804:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5a3:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.596816:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606cac00/0xef30c340bfb44690 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5a3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315f3c expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.596830:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5a2:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.596842:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606ca400/0xef30c340bfb44682 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5a2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315f0b expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.596881:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5a1:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.596895:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1c00/0xef30c340bfb44674 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5a1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315eda expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.596910:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x5a0:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.596922:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1800/0xef30c340bfb44666 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x5a0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315ea9 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.596936:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x59f:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.596947:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e0e00/0xef30c340bfb44658 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x59f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315e78 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.596960:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x59e:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.596971:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada516e1a00/0xef30c340bfb4464a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x59e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315e47 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.596985:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x59d:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.597063:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606ca800/0xef30c340bfb4463c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x59d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315e16 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.597090:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x59c:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.597105:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d4800/0xef30c340bfb4462e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x59c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315de5 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.597120:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x59b:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.597132:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60782a00/0xef30c340bfb44620 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x59b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315db4 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.597148:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x59a:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.597160:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5a00/0xef30c340bfb44612 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x59a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315d83 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.597174:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x599:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.597186:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d4a00/0xef30c340bfb44604 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x599:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315d52 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.597200:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x598:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.597211:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb445f6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x598:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315d21 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.597224:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x597:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.597236:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60315200/0xef30c340bfb445e8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x597:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315cf0 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.597248:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x596:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.597259:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45517800/0xef30c340bfb445da lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x596:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315cbf expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.597278:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x595:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.597290:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60315e00/0xef30c340bfb445cc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x595:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315c8e expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.597306:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x594:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.597320:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606caa00/0xef30c340bfb445be lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x594:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315c5d expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.597333:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x593:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.597343:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45516e00/0xef30c340bfb445b0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x593:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315c2c expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.597355:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x592:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.597365:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45516200/0xef30c340bfb445a2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x592:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315bfb expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.597377:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x591:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.597386:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45517a00/0xef30c340bfb44594 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x591:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315bca expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.597399:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x590:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.597410:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45517200/0xef30c340bfb44586 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x590:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315b99 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.597425:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x58f:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.597437:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60564400/0xef30c340bfb44578 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x58f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315b68 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.597468:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x58e:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.597482:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6032a400/0xef30c340bfb4456a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x58e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315b37 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.597502:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x58d:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.597520:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45516600/0xef30c340bfb4455c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x58d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315b06 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.597534:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x58c:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.597548:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45516000/0xef30c340bfb4454e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x58c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315ad5 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.597566:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x58b:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.597578:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45516c00/0xef30c340bfb44540 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x58b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315aa4 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.597673:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x58a:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.597689:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45516400/0xef30c340bfb44532 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x58a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315a73 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.597705:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x589:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.597719:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6032a600/0xef30c340bfb44524 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x589:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315a42 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.597736:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x588:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.597750:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6032ac00/0xef30c340bfb44516 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x588:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315a11 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.597771:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x587:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.597786:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6032a200/0xef30c340bfb44508 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x587:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83159e0 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.597800:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x586:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.597814:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60314a00/0xef30c340bfb444fa lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x586:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83159af expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.597828:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x585:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.597842:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6032a800/0xef30c340bfb444ec lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x585:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831597e expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.597879:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x584:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.597893:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45516a00/0xef30c340bfb444de lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x584:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831594d expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.597910:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x583:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.597922:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45516800/0xef30c340bfb444d0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x583:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831591c expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.597937:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x582:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.597949:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45517600/0xef30c340bfb444c2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x582:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83158eb expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.597964:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x581:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.597977:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45517e00/0xef30c340bfb444b4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x581:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83158ba expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.598066:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x580:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.598084:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45517000/0xef30c340bfb444a6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x580:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315889 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.598101:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x57f:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.598113:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada45517c00/0xef30c340bfb44498 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x57f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315858 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.598128:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x57e:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.598144:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6032be00/0xef30c340bfb4448a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x57e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315827 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.598161:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x57d:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.598176:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6032aa00/0xef30c340bfb4447c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x57d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83157f6 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.598191:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x57c:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.598203:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4556d000/0xef30c340bfb4446e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x57c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83157c5 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.598219:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x57b:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.598229:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60782400/0xef30c340bfb44460 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x57b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315794 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.598241:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x57a:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.598252:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60782800/0xef30c340bfb44452 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x57a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315763 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.598269:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x579:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.598279:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60783e00/0xef30c340bfb44444 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x579:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315732 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.598292:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x578:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.598303:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60783800/0xef30c340bfb44436 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x578:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315701 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.598316:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x577:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.598328:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60783000/0xef30c340bfb44428 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x577:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83156d0 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.598344:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x576:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.598356:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60783600/0xef30c340bfb4441a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x576:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831569f expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.598370:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x575:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.598381:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60783c00/0xef30c340bfb4440c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x575:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831566e expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.598398:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x574:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.598410:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60782c00/0xef30c340bfb443fe lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x574:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831563d expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.598423:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x573:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.598435:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6032b600/0xef30c340bfb443f0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x573:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831560c expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.598468:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x572:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.598482:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6032bc00/0xef30c340bfb443e2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x572:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83155db expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.598496:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x571:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.598510:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6032b800/0xef30c340bfb443d4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x571:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83155aa expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.598525:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x570:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.598537:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6032ae00/0xef30c340bfb443c6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x570:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315579 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.598551:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x56f:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.598563:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6032b400/0xef30c340bfb443b8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x56f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315548 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.598578:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x56e:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.598665:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6032ba00/0xef30c340bfb443aa lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x56e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315517 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.598684:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x56d:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.598696:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6032b200/0xef30c340bfb4439c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x56d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83154e6 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.598711:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x56c:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.598723:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6032a000/0xef30c340bfb4438e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x56c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83154b5 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.598740:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x56b:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.598751:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6058de00/0xef30c340bfb44380 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x56b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315484 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.598764:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x56a:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.598775:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60565c00/0xef30c340bfb44372 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x56a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315453 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.598789:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x569:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.598800:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60564c00/0xef30c340bfb44364 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x569:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315422 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.598815:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x568:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.598829:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60564200/0xef30c340bfb44356 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x568:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83153f1 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.598843:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x567:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.598879:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60564e00/0xef30c340bfb44348 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x567:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83153c0 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.598896:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x566:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.598910:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60782e00/0xef30c340bfb4433a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x566:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831538f expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.598926:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x565:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.598939:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60783400/0xef30c340bfb4432c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x565:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831535e expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.598958:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x564:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.598970:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60783a00/0xef30c340bfb4431e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x564:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831532d expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.598984:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x563:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.598995:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60564800/0xef30c340bfb44310 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x563:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83152fc expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.599025:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x562:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.599039:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60565e00/0xef30c340bfb44302 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x562:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83152cb expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.599053:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x561:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.599065:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60564a00/0xef30c340bfb442f4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x561:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831529a expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.599080:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x560:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.599092:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60565000/0xef30c340bfb442e6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x560:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315269 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.599106:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x55f:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.599117:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60565600/0xef30c340bfb442d8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x55f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315238 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.599132:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x55e:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.599145:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60783200/0xef30c340bfb442ca lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x55e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315207 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.599166:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x55d:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.599177:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60782600/0xef30c340bfb442bc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x55d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83151d6 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.599191:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x55c:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.599202:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60782000/0xef30c340bfb442ae lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x55c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83151a5 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.599216:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x55b:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.599226:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60782200/0xef30c340bfb442a0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x55b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315174 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.599239:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x55a:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.599250:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a13a00/0xef30c340bfb44292 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x55a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315143 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.599264:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x559:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.599275:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60565400/0xef30c340bfb44284 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x559:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8315112 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.599290:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x558:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.599302:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60565800/0xef30c340bfb44276 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x558:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83150e1 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.599317:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x557:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.599328:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60564600/0xef30c340bfb44268 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x557:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83150b0 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.599347:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x556:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.599359:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60565a00/0xef30c340bfb4425a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x556:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831507f expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.599373:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x555:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.599384:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60565200/0xef30c340bfb4424c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x555:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831504e expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.599398:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x554:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.599410:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60564000/0xef30c340bfb4423e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x554:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831501d expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.599423:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x553:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.599433:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada454d1400/0xef30c340bfb44230 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x553:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314fec expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.599445:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x552:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.599470:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51945a00/0xef30c340bfb44222 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x552:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314fbb expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.599486:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x551:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.599499:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6058d800/0xef30c340bfb44214 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x551:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314f8a expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.599514:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x550:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.599524:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60314e00/0xef30c340bfb44206 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x550:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314f59 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.599543:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x54f:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.599556:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a12200/0xef30c340bfb441f8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x54f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314f28 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.599570:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x54e:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.599721:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6058ca00/0xef30c340bfb441ea lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x54e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314ef7 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.599743:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x54d:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.599757:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6058d000/0xef30c340bfb441dc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x54d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314ec6 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.599773:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x54c:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.599785:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6058d600/0xef30c340bfb441ce lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x54c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314e95 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.599800:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x54b:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.599812:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6058dc00/0xef30c340bfb441c0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x54b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314e64 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.599827:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x54a:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.599838:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6058d200/0xef30c340bfb441b2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x54a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314e33 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.599877:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x549:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.599894:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a13c00/0xef30c340bfb441a4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x549:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314e02 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.599915:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x548:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.599928:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6058ce00/0xef30c340bfb44196 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x548:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314dd1 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.599942:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x547:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.599954:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6058d400/0xef30c340bfb44188 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x547:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314da0 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.599968:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x546:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.599979:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6058da00/0xef30c340bfb4417a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x546:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314d6f expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.599992:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x545:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.600069:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a12e00/0xef30c340bfb4416c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x545:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314d3e expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.600090:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x544:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.600103:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a13800/0xef30c340bfb4415e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x544:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314d0d expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.600119:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x543:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.600131:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a13400/0xef30c340bfb44150 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x543:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314cdc expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.600147:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x542:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.600160:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6058c600/0xef30c340bfb44142 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x542:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314cab expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.600183:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x541:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.600196:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6058c400/0xef30c340bfb44134 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x541:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314c7a expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.600211:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x540:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.600222:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6058c000/0xef30c340bfb44126 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x540:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314c49 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.600239:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x53f:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.600252:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6058cc00/0xef30c340bfb44118 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x53f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314c18 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.600267:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x53e:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.600278:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6058c200/0xef30c340bfb4410a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x53e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314be7 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.600293:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x53d:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.600306:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6058c800/0xef30c340bfb440fc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x53d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314bb6 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.600323:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x53c:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.600336:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada454d1200/0xef30c340bfb440ee lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x53c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314b85 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.600351:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x53b:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.600364:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada454d0000/0xef30c340bfb440e0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x53b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314b54 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.600384:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x53a:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.600399:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada454d0600/0xef30c340bfb440d2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x53a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314b23 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.600416:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x539:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.600429:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada454d0200/0xef30c340bfb440c4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x539:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314af2 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.600444:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x538:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.600470:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada454d0800/0xef30c340bfb440b6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x538:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314ac1 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.600488:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x537:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.600502:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada454d0c00/0xef30c340bfb440a8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x537:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314a90 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.600518:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x536:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.600533:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada454d0400/0xef30c340bfb4409a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x536:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314a5f expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.600549:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x535:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.600561:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada454d0a00/0xef30c340bfb4408c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x535:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314a2e expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.600578:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x534:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.600669:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a12000/0xef30c340bfb4407e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x534:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83149fd expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.600694:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x533:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.600707:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a13200/0xef30c340bfb44070 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x533:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83149cc expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.600723:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x532:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.600735:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a12600/0xef30c340bfb44062 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x532:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831499b expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.600750:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x531:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.600762:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a12400/0xef30c340bfb44054 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x531:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831496a expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.600779:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x530:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.600791:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a12c00/0xef30c340bfb44046 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x530:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314939 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.600811:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x52f:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.600823:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a13000/0xef30c340bfb44038 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x52f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314908 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.600841:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x52e:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.600877:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a12800/0xef30c340bfb4402a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x52e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83148d7 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.600895:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x52d:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.600909:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a13e00/0xef30c340bfb4401c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x52d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83148a6 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.600929:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x52c:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.600942:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a12a00/0xef30c340bfb4400e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x52c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314875 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.600958:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x52b:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.600972:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60a13600/0xef30c340bfb44000 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x52b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314844 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.600989:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x52a:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.601077:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4556d600/0xef30c340bfb43ff2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x52a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314813 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.601097:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x529:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.601111:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607e8600/0xef30c340bfb43fe4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x529:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83147e2 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.601128:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x528:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.601143:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4556ca00/0xef30c340bfb43fd6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x528:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83147b1 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.601160:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x527:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.601176:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada454d1e00/0xef30c340bfb43fc8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x527:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314780 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.601193:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x526:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.601212:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d4e00/0xef30c340bfb43fba lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x526:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831474f expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.601235:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x525:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.601249:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5000/0xef30c340bfb43fac lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x525:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831471e expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.601269:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x524:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.601286:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60315800/0xef30c340bfb43f9e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x524:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83146ed expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.601302:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x523:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.601315:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4556c400/0xef30c340bfb43f90 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x523:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83146bc expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.601331:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x522:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.601342:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4556dc00/0xef30c340bfb43f82 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x522:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831468b expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.601359:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x521:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.601373:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4556d800/0xef30c340bfb43f74 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x521:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831465a expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.601389:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x520:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.601400:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4556ce00/0xef30c340bfb43f66 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x520:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314629 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.601416:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x51f:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.601426:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4556d400/0xef30c340bfb43f58 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x51f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83145f8 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.601443:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x51e:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.601466:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4556da00/0xef30c340bfb43f4a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x51e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83145c7 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.601487:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x51d:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.601501:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4556d200/0xef30c340bfb43f3c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x51d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314596 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.601516:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x51c:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.601527:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4556c600/0xef30c340bfb43f2e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x51c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314565 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.601543:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x51b:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.601556:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada454d1c00/0xef30c340bfb43f20 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x51b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314534 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.601572:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x51a:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.601651:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada454d1000/0xef30c340bfb43f12 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x51a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314503 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.601672:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x519:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.601687:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4556c000/0xef30c340bfb43f04 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x519:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83144d2 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.601703:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x518:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.601715:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4556cc00/0xef30c340bfb43ef6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x518:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83144a1 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.601735:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x517:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.601746:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4556c200/0xef30c340bfb43ee8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x517:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314470 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.601762:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x516:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.601777:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada454d1600/0xef30c340bfb43eda lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x516:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831443f expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.601793:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x515:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.601805:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4556c800/0xef30c340bfb43ecc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x515:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831440e expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.601821:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x514:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.601833:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4556de00/0xef30c340bfb43ebe lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x514:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83143dd expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.601848:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x513:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.601888:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50a3b600/0xef30c340bfb43eb0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x513:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83143ac expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.601906:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x512:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.601917:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607e9000/0xef30c340bfb43ea2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x512:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831437b expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.601933:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x511:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.601944:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607e8e00/0xef30c340bfb43e94 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x511:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831434a expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.601966:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x510:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.601979:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada454d1800/0xef30c340bfb43e86 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x510:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314319 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.601995:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x50f:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.602080:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada454d0e00/0xef30c340bfb43e78 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x50f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83142e8 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.602098:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x50e:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.602111:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada454d1a00/0xef30c340bfb43e6a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x50e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83142b7 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.602127:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x50d:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.602139:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607e9400/0xef30c340bfb43e5c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x50d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314286 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.602155:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x50c:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.602167:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607e9a00/0xef30c340bfb43e4e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x50c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314255 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.602183:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x50b:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.602198:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51728200/0xef30c340bfb43e40 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x50b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314224 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.602215:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x50a:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.602230:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5c00/0xef30c340bfb43e32 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x50a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83141f3 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.602253:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x509:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.602266:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51729c00/0xef30c340bfb43e24 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x509:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83141c2 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.602282:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x508:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.602294:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607e9200/0xef30c340bfb43e16 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x508:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314191 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.602312:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x507:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.602325:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607e8000/0xef30c340bfb43e08 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x507:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314160 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.602342:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x506:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.602356:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51728c00/0xef30c340bfb43dfa lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x506:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831412f expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.602373:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x505:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.602386:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51729a00/0xef30c340bfb43dec lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x505:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83140fe expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.602402:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x504:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.602417:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607e8c00/0xef30c340bfb43dde lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x504:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83140cd expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.602432:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x503:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.602443:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607e8200/0xef30c340bfb43dd0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x503:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831409c expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.602474:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x502:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.602487:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607e8400/0xef30c340bfb43dc2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x502:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831406b expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.602502:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x501:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.602514:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607e8800/0xef30c340bfb43db4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x501:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831403a expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.602526:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x500:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.602539:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51729800/0xef30c340bfb43da6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x500:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8314009 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.602555:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4ff:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.602567:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607e9e00/0xef30c340bfb43d98 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4ff:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313fd8 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.602652:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4fe:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.602677:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60315400/0xef30c340bfb43d8a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4fe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313fa7 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.602697:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4fd:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.602712:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51728e00/0xef30c340bfb43d7c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4fd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313f76 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.602726:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4fc:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.602739:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51729400/0xef30c340bfb43d6e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4fc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313f45 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.602757:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4fb:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.602769:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607e8a00/0xef30c340bfb43d60 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4fb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313f14 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.602785:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4fa:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.602797:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60314800/0xef30c340bfb43d52 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4fa:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313ee3 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.602815:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4f9:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.602828:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607e9c00/0xef30c340bfb43d44 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4f9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313eb2 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.602843:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4f8:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.605116:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51729200/0xef30c340bfb43d36 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4f8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313e81 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.605174:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4f7:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.605204:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51728600/0xef30c340bfb43d28 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4f7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313e50 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.605221:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4f6:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.605235:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51728000/0xef30c340bfb43d1a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4f6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313e1f expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.605253:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4f5:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.605268:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607e9600/0xef30c340bfb43d0c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4f5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313dee expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.605293:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4f4:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.605313:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5400/0xef30c340bfb43cfe lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4f4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313dbd expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.605332:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4f3:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.605346:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607e9800/0xef30c340bfb43cf0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4f3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313d8c expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.605362:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4f2:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.605374:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d3200/0xef30c340bfb43ce2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4f2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313d5b expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.605389:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4f1:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.605403:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60314400/0xef30c340bfb43cd4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4f1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313d2a expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.605421:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4f0:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.605433:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51728800/0xef30c340bfb43cc6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4f0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313cf9 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.605462:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4ef:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.605479:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51728400/0xef30c340bfb43cb8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4ef:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313cc8 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.605496:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4ee:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.605509:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60671000/0xef30c340bfb43caa lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4ee:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313c97 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.605530:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4ed:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.605545:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51729000/0xef30c340bfb43c9c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4ed:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313c66 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.605561:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4ec:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.605577:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51729e00/0xef30c340bfb43c8e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4ec:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313c35 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.605776:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4eb:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.605796:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51728a00/0xef30c340bfb43c80 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4eb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313c04 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.605812:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4ea:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.605826:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51729600/0xef30c340bfb43c72 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4ea:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313bd3 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.605842:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4e9:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.605876:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51749200/0xef30c340bfb43c64 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4e9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313ba2 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.605894:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4e8:0x0]: losing layout lock 00010000:00010000:1.0:1788572217.605908:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50a3ae00/0xef30c340bfb43c56 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4e8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313b71 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572217.605925:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4e7:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.605999:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51748e00/0xef30c340bfb43c48 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4e7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313b40 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.606021:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4e6:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.606046:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51749400/0xef30c340bfb43c3a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4e6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313b0f expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.606062:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4e5:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.606085:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51748600/0xef30c340bfb43c2c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4e5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313ade expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.606103:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4e4:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.606116:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50a3bc00/0xef30c340bfb43c1e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4e4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313aad expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.606131:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4e3:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.606143:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50a3b400/0xef30c340bfb43c10 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4e3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313a7c expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.606160:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4e2:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.606172:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50a3b800/0xef30c340bfb43c02 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4e2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313a4b expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.606188:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4e1:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.606199:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50a3a000/0xef30c340bfb43bf4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4e1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313a1a expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.606216:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4e0:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.606228:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50a3ba00/0xef30c340bfb43be6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4e0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83139e9 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.606244:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4df:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.606255:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50a3b200/0xef30c340bfb43bd8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4df:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83139b8 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.606272:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4de:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.606286:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50a3a600/0xef30c340bfb43bca lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4de:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313987 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.606301:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4dd:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.606312:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50a3aa00/0xef30c340bfb43bbc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4dd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313956 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.606326:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4dc:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.606336:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50a3ac00/0xef30c340bfb43bae lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4dc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313925 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.606350:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4db:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.606359:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50a3a200/0xef30c340bfb43ba0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4db:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83138f4 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.606377:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4da:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.606387:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50a3a400/0xef30c340bfb43b92 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4da:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83138c3 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.606400:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4d9:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.606410:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50a3a800/0xef30c340bfb43b84 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4d9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313892 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.606424:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4d8:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.606435:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50a3be00/0xef30c340bfb43b76 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4d8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313861 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.606461:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4d7:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.606475:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada50a3b000/0xef30c340bfb43b68 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4d7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313830 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.606491:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4d6:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.606501:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d2600/0xef30c340bfb43b5a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4d6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83137ff expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.606516:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4d5:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.606525:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d2800/0xef30c340bfb43b4c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4d5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83137ce expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.606540:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4d4:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.606550:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d3400/0xef30c340bfb43b3e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4d4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831379d expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.606567:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4d3:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.606577:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d2000/0xef30c340bfb43b30 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4d3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831376c expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.606592:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4d2:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.606603:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d2c00/0xef30c340bfb43b22 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4d2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831373b expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.606618:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4d1:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.606630:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d2200/0xef30c340bfb43b14 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4d1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831370a expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.606646:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4d0:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.606659:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d2400/0xef30c340bfb43b06 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4d0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83136d9 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.606675:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4cf:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.606688:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d3e00/0xef30c340bfb43af8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4cf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83136a8 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.606834:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4ce:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.606890:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51749a00/0xef30c340bfb43aea lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4ce:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313677 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.606908:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4cd:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.606921:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51749e00/0xef30c340bfb43adc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4cd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313646 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.606939:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4cc:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.606951:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51748000/0xef30c340bfb43ace lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4cc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313615 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.606965:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4cb:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.606976:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51748c00/0xef30c340bfb43ac0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4cb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83135e4 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.606989:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4ca:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.606998:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51748200/0xef30c340bfb43ab2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4ca:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83135b3 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.607014:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4c9:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.607025:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51748400/0xef30c340bfb43aa4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4c9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313582 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.607039:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4c8:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.607050:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51748800/0xef30c340bfb43a96 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4c8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313551 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.607063:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4c7:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.607181:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51749800/0xef30c340bfb43a88 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4c7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313520 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.607197:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4c6:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.607208:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51748a00/0xef30c340bfb43a7a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4c6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83134ef expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.607226:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4c5:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.607236:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51749000/0xef30c340bfb43a6c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4c5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83134be expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.607251:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4c4:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.607263:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d2a00/0xef30c340bfb43a5e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4c4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831348d expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.607277:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4c3:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.607288:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d3000/0xef30c340bfb43a50 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4c3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831345c expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.607302:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4c2:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.607312:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d3600/0xef30c340bfb43a42 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4c2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831342b expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.607326:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4c1:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.607335:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d3c00/0xef30c340bfb43a34 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4c1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83133fa expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.607348:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4c0:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.607359:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d3800/0xef30c340bfb43a26 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4c0:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83133c9 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.607372:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4bf:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.607382:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d2e00/0xef30c340bfb43a18 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4bf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313398 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.607400:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4be:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.607410:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d3a00/0xef30c340bfb43a0a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4be:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313367 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.607422:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4bd:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.607433:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51749600/0xef30c340bfb439fc lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4bd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313336 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.607446:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4bc:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.607468:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51749c00/0xef30c340bfb439ee lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4bc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313305 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.607483:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4bb:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.607494:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51945c00/0xef30c340bfb439e0 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4bb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83132d4 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.607507:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4ba:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.607517:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51944600/0xef30c340bfb439d2 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4ba:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83132a3 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.607530:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4b9:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.607544:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51945400/0xef30c340bfb439c4 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4b9:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313272 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.607557:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4b8:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.607567:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51945000/0xef30c340bfb439b6 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4b8:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313241 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.607585:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4b7:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.607598:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60670200/0xef30c340bfb439a8 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4b7:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8313210 expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.607615:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4b6:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.607628:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60671600/0xef30c340bfb4399a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4b6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83131df expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.607644:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4b5:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.607658:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada60670800/0xef30c340bfb4398c lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4b5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83131ae expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.607674:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4b4:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.607686:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51945200/0xef30c340bfb4397e lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4b4:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831317d expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.608998:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4b3:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.609051:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51944000/0xef30c340bfb43970 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4b3:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831314c expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.609256:0:86341:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x4b2:0x0]: losing layout lock 00010000:00010000:0.0:1788572217.609274:0:86341:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51944c00/0xef30c340bfb43962 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x4b2:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b831311b expref: -99 pid: 85679 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788572217.609288:0:86341:0:(ldlm_request.c:2901:ldlm_cancel_unused_locks_for_replay()) Canceled 0 unused locks from namespace lustre-MDT0000-mdc-ffff9ada58b0a000 00000100:00080000:0.0:1788572217.610153:0:86341:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from REPLAY_LOCKS to REPLAY_WAIT 00000100:00100000:3.0:1788572217.610259:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60535f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615202176:192.168.202.113@tcp:400:ldlm_lock_repla.0 00000100:00100000:0.0:1788572217.668739:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 214748367202 gen 9 00000100:00080000:0.0:1788572217.670143:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from REPLAY_WAIT to RECOVER 00000100:00080000:0.0:1788572217.670167:0:2382:0:(client.c:3248:ptlrpc_resend_req()) @@@ going to resend req@ffff9ada60535f80 x1875449615202176/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572233 ref 1 fl Interpret:RQU/2c0/0 rc 0/0 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.670182:0:2382:0:(client.c:3257:ptlrpc_resend_req()) @@@ it has reply, so skip it req@ffff9ada60535f80 x1875449615202176/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572233 ref 1 fl Interpret:RQU/2c0/0 rc 0/0 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572217.670190:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from RECOVER to FULL 00000100:02000000:2.0:1788572217.670969:0:2382:0:(import.c:1787:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff9ada58b0a000: Connection restored to 192.168.202.113@tcp (at 192.168.202.113@tcp) 00000100:00100000:2.0:1788572217.670979:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60535f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615202176:192.168.202.113@tcp:400:ldlm_lock_repla.0 00000080:00200008:3.0:1788572217.880282:0:86348:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1788572217.880301:0:86348:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:3.0:1788572217.880333:0:86348:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1788572217.880406:0:86348:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51786e00/0xef30c340bfb45a5c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 86348 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572217.880418:0:86348:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51786e00/0xef30c340bfb45a5c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 86348 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572217.880429:0:86348:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51786e00/0xef30c340bfb45a5c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 86348 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572217.880470:0:86348:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517a0380 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:86348:1875449615202304:192.168.202.113@tcp:101:lfs.0 00000100:00100000:3.0:1788572217.880650:0:86348:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada51964d80 going to sleep for 11 seconds 00000100:00100000:0.0:1788572217.890280:0:86348:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517a0380 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:86348:1875449615202304:192.168.202.113@tcp:101:lfs.0 00010000:00010000:0.0:1788572217.890317:0:86348:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51786e00/0xef30c340bfb45a5c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b831eddc expref: -99 pid: 86348 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572217.890331:0:86348:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51786e00/0xef30c340bfb45a5c lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b831eddc expref: -99 pid: 86348 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572217.890347:0:86348:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51786e00/0xef30c340bfb45a5c lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b831eddc expref: -99 pid: 86348 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572217.890360:0:86348:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51786e00/0xef30c340bfb45a5c lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b831eddc expref: -99 pid: 86348 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572217.890377:0:86348:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51786e00/0xef30c340bfb45a5c lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b831eddc expref: -99 pid: 86348 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572217.890386:0:86348:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51786e00/0xef30c340bfb45a5c lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b831eddc expref: -99 pid: 86348 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572217.890394:0:86348:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51786e00/0xef30c340bfb45a5c lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b831eddc expref: -99 pid: 86348 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788572217.890411:0:86348:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada517a0380 x1875449615202304/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788572233 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788572217.890426:0:86348:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51786e00/0xef30c340bfb45a5c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b831eddc expref: -99 pid: 86348 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572217.890451:0:86348:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1788572217.890461:0:86348:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1788572217.890496:0:86348:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:0.0:1788572217.890503:0:86348:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb45a5c 00000080:00010000:0.0:1788572217.890517:0:86348:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572217.890521:0:86348:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb45a5c from it ffffab0ec0d4bcb8 00010000:00010000:0.0:1788572217.890524:0:86348:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51786e00/0xef30c340bfb45a5c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b831eddc expref: -99 pid: 86348 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572217.890533:0:86348:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51786e00/0xef30c340bfb45a5c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b831eddc expref: -99 pid: 86348 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:0.0:1788572217.894215:0:86348:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1788572217.894228:0:86348:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572217.894261:0:86348:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572217.894287:0:86348:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51786e00/0xef30c340bfb45a5c lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b831eddc expref: -99 pid: 86348 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788572217.894308:0:86348:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51786e00/0xef30c340bfb45a5c lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b831eddc expref: -99 pid: 86348 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572217.894322:0:86348:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572217.894332:0:86348:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572217.894336:0:86348:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb45a5c 00000080:00010000:0.0:1788572217.894345:0:86348:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb45a5c from it ffffab0ec0d4bcb8 00010000:00010000:0.0:1788572217.894349:0:86348:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51786e00/0xef30c340bfb45a5c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b831eddc expref: -99 pid: 86348 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788572217.894359:0:86348:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51786e00/0xef30c340bfb45a5c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b831eddc expref: -99 pid: 86348 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788572217.894564:0:86348:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572217.894574:0:86348:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572217.894590:0:86348:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51786e00/0xef30c340bfb45a5c lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b831eddc expref: -99 pid: 86348 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788572217.894599:0:86348:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51786e00/0xef30c340bfb45a5c lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b831eddc expref: -99 pid: 86348 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788572217.894606:0:86348:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572217.894610:0:86348:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572217.894613:0:86348:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb45a5c 00000080:00010000:0.0:1788572217.894617:0:86348:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb45a5c from it ffffab0ec0d4bbb0 00010000:00010000:0.0:1788572217.894620:0:86348:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51786e00/0xef30c340bfb45a5c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b831eddc expref: -99 pid: 86348 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788572217.894626:0:86348:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51786e00/0xef30c340bfb45a5c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b831eddc expref: -99 pid: 86348 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00000080:00200000:0.0:1788572217.894635:0:86348:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:0.0:1788572217.894645:0:86348:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200080:0.0:1788572217.894768:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.894787:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45561800 uarg=0000000000000000 00800000:00000004:0.0:1788572217.894795:0:86348:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: age 4070, max_age 4134 00000100:00100000:0.0:1788572217.894892:0:86348:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60537800 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:86348:1875449615202432:192.168.202.113@tcp:41:lfs.0 00000100:00100000:0.0:1788572217.894993:0:86348:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60a77900 going to sleep for 11 seconds 00000100:00100000:1.0:1788572217.898895:0:86348:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60537800 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:86348:1875449615202432:192.168.202.113@tcp:41:lfs.0 00800000:00000004:1.0:1788572217.898933:0:86348:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: update ffff9ada4481d198 cache blocks 317317/353529 objects 1023180/1024000 00000080:00200080:2.0:1788572217.899944:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:2.0:1788572217.899969:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6000 uarg=0000000000000000 00800000:00000004:2.0:1788572217.899978:0:86348:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: age 4070, max_age 4134 00000100:00100000:2.0:1788572217.900038:0:86348:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517ff800 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:86348:1875449615202560:192.168.202.113@tcp:41:lfs.0 00000100:00100000:2.0:1788572217.900341:0:86348:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2d80 going to sleep for 11 seconds 00000100:00100000:0.0:1788572217.903299:0:86348:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517ff800 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:86348:1875449615202560:192.168.202.113@tcp:41:lfs.0 00800000:00000004:0.0:1788572217.903335:0:86348:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: update ffff9ada44819e18 cache blocks 317392/353529 objects 1023736/1024000 00000080:00200080:0.0:1788572217.903421:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.903452:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.903462:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.903467:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.903471:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.903475:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.903478:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.903482:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.903486:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.903490:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.903493:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.903497:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.903501:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.903957:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.903970:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.903977:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.903981:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.903984:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.903987:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.903990:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.903994:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.903997:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904001:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904004:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904008:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904011:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904015:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904018:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904252:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904259:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904264:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904268:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904272:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904276:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904279:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904283:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904287:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904290:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904294:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904298:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904302:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904305:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904309:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904313:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904317:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904321:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904325:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904328:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904332:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904335:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904338:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904341:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904345:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904348:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904351:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904354:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904357:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904359:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904362:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904365:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904368:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904371:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904374:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904377:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904381:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904384:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904388:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904391:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904395:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904398:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904401:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904404:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904408:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904411:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904415:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904418:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904422:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904425:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904429:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904433:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904437:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904440:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904455:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904459:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904463:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904466:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904469:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904472:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904475:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904479:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904482:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904485:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904488:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904491:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904494:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904497:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904500:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904503:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904513:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904516:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904520:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904522:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904525:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904528:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904531:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904534:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904537:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904540:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904543:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904546:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904549:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904551:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904554:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904556:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904560:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904563:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904566:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904568:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.904571:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.904574:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.905464:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.905479:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.905486:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.905490:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.905494:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00800000:00000080:0.0:1788572217.905497:0:86348:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00000080:00200080:0.0:1788572217.905501:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:0.0:1788572217.905569:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00020000:00000004:0.0:1788572217.905646:0:86348:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: age 4070, max_age 4134 00000100:00100000:0.0:1788572217.905706:0:86348:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60534a80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:86348:1875449615202688:192.168.202.113@tcp:13:lfs.0 00000100:00100000:0.0:1788572217.905812:0:86348:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60a77a80 going to sleep for 11 seconds 00000100:00100000:0.0:1788572217.911555:0:86348:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60534a80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:86348:1875449615202688:192.168.202.113@tcp:13:lfs.0 00020000:00000004:0.0:1788572217.911595:0:86348:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: update ffff9ada4354d198 cache blocks 901357/958279 objects 261523/262144 00000080:00200080:0.0:1788572217.911745:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:0.0:1788572217.911768:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45563400 uarg=0000000000000000 00020000:00000004:0.0:1788572217.911777:0:86348:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: age 4070, max_age 4134 00000100:00100000:0.0:1788572217.911824:0:86348:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60534a80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:86348:1875449615202816:192.168.202.113@tcp:13:lfs.0 00000100:00100000:0.0:1788572217.911956:0:86348:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60a77a80 going to sleep for 11 seconds 00000100:00100000:0.0:1788572217.916961:0:86348:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60534a80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:86348:1875449615202816:192.168.202.113@tcp:13:lfs.0 00020000:00000004:0.0:1788572217.918195:0:86348:0:(obd_class.h:1116:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: update ffff9ada43549e18 cache blocks 901363/958279 objects 261526/262144 00000080:00200080:2.0:1788572217.918705:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.918736:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.918747:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.918763:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.918770:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.918775:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.918780:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.918784:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.918788:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.918791:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.918796:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.918799:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.918803:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.918807:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.918812:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.918815:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.918819:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.918823:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.918827:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.918830:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.918834:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.918837:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.918841:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.918846:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.918850:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.918900:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.918908:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.918912:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.918916:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.918920:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.918924:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.918928:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.918932:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.918937:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.918941:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.918944:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.918949:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.918952:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.918957:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.918961:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.918965:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.918968:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.918973:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.918977:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.918982:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.918985:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.918990:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.919005:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.919011:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.919015:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.919020:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.919024:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.919028:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.919032:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.919036:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.919040:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.919044:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.919047:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.919051:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.919055:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.919058:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.919070:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.919075:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.919079:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.919083:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.919087:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.919092:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.919095:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.919099:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.919102:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.919105:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.919109:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.919112:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.919115:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.919119:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.919123:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.919127:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.919131:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.919135:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.919139:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.919144:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.919147:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.919152:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.919156:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.919160:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.919163:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.919168:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.919171:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.919175:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.919179:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.919183:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.919186:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.919192:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.919195:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.919199:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.919203:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.919207:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.919211:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.919215:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.919219:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.919222:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.919226:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.919230:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.919233:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.919236:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.919239:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.919243:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.919247:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.919250:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.919253:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.919257:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.919261:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.919274:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.919279:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.919283:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.919287:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.919291:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.919295:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.919299:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.919303:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.919309:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.919314:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.919318:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.919322:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200080:2.0:1788572217.919326:0:86348:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd02536f40 00020000:00000080:2.0:1788572217.919329:0:86348:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d6400 uarg=0000000000000000 00000080:00200008:2.0:1788572217.919348:0:86348:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00010000:00010000:1.0:1788572218.231989:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572219.264027:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788572219.768113:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449615202944:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572219.768238:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615203200:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572219.768269:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615203456:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572219.768303:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1788572219.768310:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615203072:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572219.768331:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517ff480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615203328:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572219.779235:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615203072:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572219.779255:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517ff480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615203328:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572219.779358:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615203200:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572219.779373:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615203456:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572219.779899:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517ffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449615202944:192.168.202.113@tcp:400:kworker.0 00010000:00010000:3.0:1788572220.280561:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572221.306320:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572222.327438:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572223.352789:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1788572223.954636:0:86629:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 00010000:00010000:1.0:1788572224.377110:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572224.888266:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada44fe8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449615203712:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572224.888373:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada44fe8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449615203968:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572224.888412:0:2383:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:1.0:1788572224.888419:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada44feb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449615203584:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572224.888448:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada44fe9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449615203840:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572224.888471:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada44feaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449615204096:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572224.898994:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada44fe9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449615203840:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572224.899019:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada44fe8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449615203712:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572224.900904:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada44feb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449615203584:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572224.900925:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada44feaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449615204096:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572224.900938:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada44fe8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449615203968:192.168.202.113@tcp:400:kworker.0 00010000:00010000:2.0:1788572225.402091:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1788572226.105555:0:86777:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec 00010000:00010000:3.0:1788572226.427107:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572227.449180:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:2.0:1788572228.060845:0:86816:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1788572228.060886:0:86816:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788572228.060917:0:86816:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788572228.060943:0:86816:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51786e00/0xef30c340bfb45a5c lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b831eddc expref: -99 pid: 86348 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00010000:00010000:2.0:1788572228.060959:0:86816:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51786e00/0xef30c340bfb45a5c lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b831eddc expref: -99 pid: 86348 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00000080:00200000:2.0:1788572228.060967:0:86816:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788572228.060975:0:86816:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788572228.060979:0:86816:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb45a5c 00000080:00010000:2.0:1788572228.060988:0:86816:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb45a5c from it ffffab0ec0d7fcb8 00010000:00010000:2.0:1788572228.060992:0:86816:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51786e00/0xef30c340bfb45a5c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b831eddc expref: -99 pid: 86348 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00010000:00010000:2.0:1788572228.061000:0:86816:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51786e00/0xef30c340bfb45a5c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b831eddc expref: -99 pid: 86348 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00200008:2.0:1788572228.061590:0:86816:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 0, flags 0, foreign 0 00000080:00200000:2.0:1788572228.061596:0:86816:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788572228.061608:0:86816:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788572228.061617:0:86816:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51786e00/0xef30c340bfb45a5c lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b831eddc expref: -99 pid: 86348 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:2.0:1788572228.061626:0:86816:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51786e00/0xef30c340bfb45a5c lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b831eddc expref: -99 pid: 86348 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00200000:2.0:1788572228.061632:0:86816:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788572228.061637:0:86816:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788572228.061640:0:86816:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb45a5c 00000080:00010000:2.0:1788572228.061644:0:86816:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb45a5c from it ffffab0ec0d7fcb8 00010000:00010000:2.0:1788572228.061647:0:86816:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51786e00/0xef30c340bfb45a5c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b831eddc expref: -99 pid: 86348 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:2.0:1788572228.061655:0:86816:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51786e00/0xef30c340bfb45a5c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b831eddc expref: -99 pid: 86348 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:2.0:1788572228.061747:0:86816:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788572228.061754:0:86816:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788572228.061761:0:86816:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51786e00/0xef30c340bfb45a5c lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b831eddc expref: -99 pid: 86348 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:2.0:1788572228.061770:0:86816:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51786e00/0xef30c340bfb45a5c lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b831eddc expref: -99 pid: 86348 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:2.0:1788572228.061783:0:86816:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788572228.061787:0:86816:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788572228.061789:0:86816:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb45a5c 00000080:00010000:2.0:1788572228.061793:0:86816:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb45a5c from it ffffab0ec0d7fbb0 00010000:00010000:2.0:1788572228.061797:0:86816:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51786e00/0xef30c340bfb45a5c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b831eddc expref: -99 pid: 86348 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:2.0:1788572228.061805:0:86816:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51786e00/0xef30c340bfb45a5c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b831eddc expref: -99 pid: 86348 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000080:00200000:2.0:1788572228.061813:0:86816:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:2.0:1788572228.061819:0:86816:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200080:2.0:1788572228.062148:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:2.0:1788572228.062163:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada507d7800 uarg=0000000000000000 00800000:00000004:2.0:1788572228.062171:0:86816:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: age 4135, max_age 4144 00000100:00100000:2.0:1788572228.062217:0:86816:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517ffb80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:86816:1875449615204224:192.168.202.113@tcp:41:lfs.0 00000100:00100000:2.0:1788572228.066651:0:86816:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2e40 going to sleep for 11 seconds 00000100:00100000:0.0:1788572228.079586:0:86816:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517ffb80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:86816:1875449615204224:192.168.202.113@tcp:41:lfs.0 00800000:00000004:0.0:1788572228.079619:0:86816:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: update ffff9ada4481d198 cache blocks 317317/353529 objects 1023288/1024000 00000080:00200080:0.0:1788572228.079707:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.079725:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00800000:00000004:0.0:1788572228.079733:0:86816:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: age 4135, max_age 4144 00000100:00100000:0.0:1788572228.079770:0:86816:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60534a80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:86816:1875449615204352:192.168.202.113@tcp:41:lfs.0 00000100:00100000:0.0:1788572228.080520:0:86816:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada51964180 going to sleep for 11 seconds 00000100:00100000:0.0:1788572228.093343:0:86816:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60534a80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:86816:1875449615204352:192.168.202.113@tcp:41:lfs.0 00800000:00000004:0.0:1788572228.093377:0:86816:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: update ffff9ada44819e18 cache blocks 317392/353529 objects 1023736/1024000 00000080:00200080:0.0:1788572228.093519:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093540:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093548:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093554:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093558:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093562:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093565:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093568:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093572:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093575:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093579:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093583:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093586:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093590:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093593:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093597:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093600:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093603:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093607:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093610:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093614:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093617:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093620:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093624:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093627:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093631:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093634:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093637:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093642:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093645:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093649:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093653:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093657:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093660:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093664:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093668:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093672:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093676:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093679:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093683:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093687:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093690:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093694:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093697:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093701:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093704:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093708:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093712:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093717:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093720:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093725:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093728:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093732:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093736:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093739:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093743:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093747:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093750:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093753:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093756:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093760:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093763:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093767:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093781:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093787:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093791:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093795:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093799:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093802:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093806:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093811:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093814:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093819:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093822:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093825:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093829:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093833:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093836:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093841:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093845:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093849:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093894:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093902:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093907:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093911:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093915:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093920:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093924:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093927:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093931:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093936:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093939:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093943:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093946:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093951:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093954:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093959:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093962:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093967:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093970:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093974:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093977:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093981:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093984:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093988:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093991:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.093994:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.093998:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.094020:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.094024:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.094028:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.094032:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.094035:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.094039:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.094042:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.094425:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.094433:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.094437:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.094441:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.094444:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.094448:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.094452:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.094456:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.094459:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.094463:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00800000:00000080:0.0:1788572228.094466:0:86816:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.094470:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.094477:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00020000:00000004:0.0:1788572228.094486:0:86816:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: age 4135, max_age 4144 00000100:00100000:0.0:1788572228.094533:0:86816:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60534a80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:86816:1875449615204480:192.168.202.113@tcp:13:lfs.0 00000100:00100000:0.0:1788572228.094636:0:86816:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada51964180 going to sleep for 11 seconds 00000100:00100000:0.0:1788572228.102682:0:86816:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60534a80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:86816:1875449615204480:192.168.202.113@tcp:13:lfs.0 00020000:00000004:0.0:1788572228.102715:0:86816:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: update ffff9ada4354d198 cache blocks 901357/958279 objects 261529/262144 00000080:00200080:0.0:1788572228.102797:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.102817:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00020000:00000004:0.0:1788572228.102826:0:86816:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: age 4135, max_age 4144 00000100:00100000:0.0:1788572228.102894:0:86816:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60534a80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:86816:1875449615204608:192.168.202.113@tcp:13:lfs.0 00000100:00100000:0.0:1788572228.102973:0:86816:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada51964180 going to sleep for 11 seconds 00000100:00100000:0.0:1788572228.116580:0:86816:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60534a80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:86816:1875449615204608:192.168.202.113@tcp:13:lfs.0 00020000:00000004:0.0:1788572228.116615:0:86816:0:(obd_class.h:1116:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: update ffff9ada43549e18 cache blocks 901363/958279 objects 261534/262144 00000080:00200080:0.0:1788572228.116713:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.116737:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.116745:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.116749:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.116753:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.116756:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.116760:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.116763:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.116767:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117255:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117268:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117276:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117282:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117297:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117302:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117306:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117310:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117313:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117317:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117320:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117324:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117327:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117331:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117334:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117338:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117341:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117345:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117348:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117351:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117354:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117357:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117361:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117364:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117367:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117370:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117374:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117377:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117381:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117384:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117387:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117391:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117394:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117397:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117401:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117404:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117407:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117411:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117414:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117418:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117422:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117425:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117429:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117433:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117436:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117439:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117442:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117446:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117449:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117452:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117456:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117459:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117462:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117467:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117470:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117473:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117477:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117480:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117483:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117487:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117491:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117495:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117498:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117502:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117505:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117512:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117516:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117519:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117522:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117526:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117530:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117534:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117536:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117541:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117544:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117548:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117551:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117554:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117558:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117562:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117565:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117569:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117572:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117576:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117579:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117583:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117587:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117591:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117594:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117598:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117601:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117604:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117607:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117611:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117614:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117618:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117621:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117625:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117628:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117632:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117635:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117639:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117642:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117646:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117650:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117654:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117657:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117661:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117664:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117669:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117672:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117675:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117678:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117682:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117685:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200080:0.0:1788572228.117688:0:86816:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff0470e600 00020000:00000080:0.0:1788572228.117692:0:86816:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45560c00 uarg=0000000000000000 00000080:00200008:0.0:1788572228.117707:0:86816:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00010000:00010000:2.0:1788572228.477038:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572229.500156:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572230.010238:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60534e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615204864:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572230.010358:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60536680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615205120:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572230.010399:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:0.0:1788572230.010405:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60534a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449615204736:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572230.010437:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60536a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615204992:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572230.010463:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60534380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615205248:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572230.018509:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60534a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449615204736:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572230.018754:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60536680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615205120:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572230.039758:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60536a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615204992:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572230.039778:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60534380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615205248:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572230.039789:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60534e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615204864:192.168.202.113@tcp:400:kworker.0 00010000:00010000:1.0:1788572230.522134:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572237.692287:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572238.711076:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572239.735462:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572240.247441:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45411c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449615410560:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572240.252919:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45410a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449615410688:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572240.253037:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45410000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449615410944:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572240.253071:0:2383:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1788572240.253080:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45413800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449615410816:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572240.253108:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45413b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449615411072:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572240.260228:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45411c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449615410560:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572240.260322:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45413b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449615411072:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572240.263380:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45410a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449615410688:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572240.263404:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45410000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449615410944:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572240.263682:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45413800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449615410816:192.168.202.113@tcp:400:kworker.0 00000080:00200000:3.0:1788572240.364053:0:85409:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:3.0:1788572240.364091:0:85409:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1788572240.364183:0:85409:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787800/0xef30c340bfb4703c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 85409 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572240.364195:0:85409:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787800/0xef30c340bfb4703c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 85409 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572240.364207:0:85409:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787800/0xef30c340bfb4703c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 85409 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572240.364243:0:85409:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517a3b80 pname:cluuid:pid:xid:nid:opc:job bash:35eb2d60-2de0-4b00-9815-d5153edd0057:85409:1875449615411200:192.168.202.113@tcp:101:bash.0 00000100:00100000:3.0:1788572240.364365:0:85409:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60419e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788572240.369373:0:85409:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517a3b80 pname:cluuid:pid:xid:nid:opc:job bash:35eb2d60-2de0-4b00-9815-d5153edd0057:85409:1875449615411200:192.168.202.113@tcp:101:bash.0 00010000:00010000:1.0:1788572240.369413:0:85409:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787800/0xef30c340bfb4703c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b832637e expref: -99 pid: 85409 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572240.369428:0:85409:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787800/0xef30c340bfb4703c lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b832637e expref: -99 pid: 85409 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572240.369442:0:85409:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787800/0xef30c340bfb4703c lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b832637e expref: -99 pid: 85409 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572240.369455:0:85409:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787800/0xef30c340bfb4703c lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b832637e expref: -99 pid: 85409 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572240.369470:0:85409:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787800/0xef30c340bfb4703c lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b832637e expref: -99 pid: 85409 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572240.369478:0:85409:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787800/0xef30c340bfb4703c lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b832637e expref: -99 pid: 85409 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572240.369486:0:85409:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787800/0xef30c340bfb4703c lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b832637e expref: -99 pid: 85409 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1788572240.369499:0:85409:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada517a3b80 x1875449615411200/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788572256 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1788572240.369521:0:85409:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787800/0xef30c340bfb4703c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b832637e expref: -99 pid: 85409 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572240.369535:0:85409:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1788572240.369543:0:85409:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1788572240.369579:0:85409:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:1.0:1788572240.369586:0:85409:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4703c 00000080:00010000:1.0:1788572240.369592:0:85409:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:1.0:1788572240.369596:0:85409:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4703c from it ffffab0ec17c3bb0 00010000:00010000:1.0:1788572240.369599:0:85409:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787800/0xef30c340bfb4703c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b832637e expref: -99 pid: 85409 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572240.369609:0:85409:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787800/0xef30c340bfb4703c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b832637e expref: -99 pid: 85409 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572240.369649:0:85409:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:1.0:1788572240.369666:0:85409:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 304000 00000080:00200008:1.0:1788572240.369762:0:85409:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000002:00010000:1.0:1788572240.369793:0:85409:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: readdir flags 0 00010000:00010000:1.0:1788572240.369803:0:85409:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787800/0xef30c340bfb4703c lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b832637e expref: -99 pid: 85409 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1788572240.369816:0:85409:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787800/0xef30c340bfb4703c lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b832637e expref: -99 pid: 85409 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000100:00100000:1.0:1788572240.371166:0:85409:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45413800 pname:cluuid:pid:xid:nid:opc:job bash:35eb2d60-2de0-4b00-9815-d5153edd0057:85409:1875449615411328:192.168.202.113@tcp:37:bash.0 00000100:00100000:1.0:1788572240.371311:0:85409:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada4522e6c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572240.390977:0:85409:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45413800 pname:cluuid:pid:xid:nid:opc:job bash:35eb2d60-2de0-4b00-9815-d5153edd0057:85409:1875449615411328:192.168.202.113@tcp:37:bash.0 00010000:00010000:0.0:1788572240.392457:0:85409:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787800/0xef30c340bfb4703c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b832637e expref: -99 pid: 85409 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788572240.392484:0:85409:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787800/0xef30c340bfb4703c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b832637e expref: -99 pid: 85409 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788572240.392528:0:85409:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572240.392558:0:85409:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572240.392578:0:85409:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787800/0xef30c340bfb4703c lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b832637e expref: -99 pid: 85409 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788572240.392593:0:85409:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787800/0xef30c340bfb4703c lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b832637e expref: -99 pid: 85409 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788572240.392603:0:85409:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572240.392610:0:85409:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572240.392614:0:85409:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4703c 00000080:00010000:0.0:1788572240.392620:0:85409:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4703c from it ffffab0ec17c3cd8 00010000:00010000:0.0:1788572240.392624:0:85409:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787800/0xef30c340bfb4703c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b832637e expref: -99 pid: 85409 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788572240.392631:0:85409:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787800/0xef30c340bfb4703c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b832637e expref: -99 pid: 85409 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00000080:00200000:0.0:1788572240.393084:0:85409:0:(dir.c:332:ll_iterate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) pos/size0/69632 32bit_api 0 00000080:00200000:0.0:1788572240.393149:0:85409:0:(dir.c:332:ll_iterate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) pos/size18446744073709551614/69632 32bit_api 0 00000080:00200008:0.0:1788572240.393181:0:85409:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 304000 00000080:00200000:3.0:1788572240.402299:0:87034:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:3.0:1788572240.405933:0:87034:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1788572240.405967:0:87034:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787800/0xef30c340bfb4703c lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b832637e expref: -99 pid: 85409 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00010000:00010000:3.0:1788572240.405987:0:87034:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787800/0xef30c340bfb4703c lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b832637e expref: -99 pid: 85409 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00000080:00200000:3.0:1788572240.405997:0:87034:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1788572240.406007:0:87034:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:3.0:1788572240.406012:0:87034:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4703c 00000080:00010000:3.0:1788572240.406021:0:87034:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4703c from it ffffab0ec15d3b58 00010000:00010000:3.0:1788572240.406026:0:87034:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787800/0xef30c340bfb4703c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b832637e expref: -99 pid: 85409 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00010000:00010000:3.0:1788572240.406036:0:87034:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787800/0xef30c340bfb4703c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b832637e expref: -99 pid: 85409 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00200000:3.0:1788572240.406056:0:87034:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:3.0:1788572240.406080:0:87034:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f39.replay-single-*, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), flags=8 00000080:00200000:3.0:1788572240.406086:0:87034:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f39.replay-single-*, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=getattr 00000002:00010000:3.0:1788572240.406103:0:87034:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f39.replay-single-*,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 01 00010000:00010000:3.0:1788572240.406161:0:87034:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787e00/0xef30c340bfb47043 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 87034 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572240.406170:0:87034:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787e00/0xef30c340bfb47043 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 87034 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572240.406192:0:87034:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787e00/0xef30c340bfb47043 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 87034 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572240.406226:0:87034:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517a3b80 pname:cluuid:pid:xid:nid:opc:job checkstat:35eb2d60-2de0-4b00-9815-d5153edd0057:87034:1875449615411456:192.168.202.113@tcp:101:checkstat.0 00000100:00100000:3.0:1788572240.406346:0:87034:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60419cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572240.413704:0:87034:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517a3b80 pname:cluuid:pid:xid:nid:opc:job checkstat:35eb2d60-2de0-4b00-9815-d5153edd0057:87034:1875449615411456:192.168.202.113@tcp:101:checkstat.0 00010000:00010000:0.0:1788572240.413734:0:87034:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787e00/0xef30c340bfb47043 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 87034 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572240.413750:0:87034:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787e00/0xef30c340bfb47043 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 87034 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572240.413758:0:87034:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787e00/0xef30c340bfb47043 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 87034 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572240.413765:0:87034:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787e00/0xef30c340bfb47043 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 87034 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572240.413775:0:87034:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787e00/0xef30c340bfb47043 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 87034 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572240.413781:0:87034:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada51787e00 already unused, calling callback (ffffffffc1966630) 00010000:00010000:0.0:1788572240.413788:0:87034:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787e00/0xef30c340bfb47043 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 87034 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572240.413809:0:87034:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787e00/0xef30c340bfb47043 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 87034 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572240.413817:0:87034:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787e00/0xef30c340bfb47043 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 87034 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572240.413824:0:87034:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1788572240.413831:0:87034:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787e00/0xef30c340bfb47043 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 87034 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572240.413838:0:87034:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787e00/0xef30c340bfb47043 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 87034 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788572240.413899:0:87034:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9ada517a3b80 x1875449615411456/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/584 e 0 to 0 dl 1788572256 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'checkstat.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1788572240.413916:0:87034:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1788572240.413951:0:87034:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787800/0xef30c340bfb4703c lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b832637e expref: -99 pid: 85409 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:0.0:1788572240.413966:0:87034:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787800/0xef30c340bfb4703c lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b832637e expref: -99 pid: 85409 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00010000:0.0:1788572240.413976:0:87034:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4703c from it ffffab0ec15d3930 00010000:00010000:0.0:1788572240.413980:0:87034:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787800/0xef30c340bfb4703c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b832637e expref: -99 pid: 85409 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:0.0:1788572240.413988:0:87034:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787800/0xef30c340bfb4703c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b832637e expref: -99 pid: 85409 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:0.0:1788572240.760049:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572241.785900:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572242.811043:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572243.831459:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572244.856989:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572245.369057:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45410000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449615411584:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572245.369174:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45413b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449615411840:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572245.369205:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45412d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449615412096:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572245.369237:0:2383:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1788572245.369244:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45410a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449615411712:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572245.369268:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45412300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449615411968:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572245.374328:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45412300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449615411968:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572245.374350:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45412d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449615412096:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572245.391429:0:2383:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 219043332496 gen 9 00000100:00100000:1.0:1788572245.391621:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45410a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449615411712:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572245.391638:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45410000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449615411584:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572245.391651:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45413b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449615411840:192.168.202.113@tcp:400:kworker.0 00010000:00010000:1.0:1788572245.882001:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572246.903139:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572247.927815:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572248.953018:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1788572249.163942:0:87599:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == replay-single test 41: read from a valid osc while other oscs are invalid ========================================================== 21:37:27 (1788572247) 00000080:00200000:2.0:1788572249.252326:0:87602:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788572249.252363:0:87602:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788572249.252385:0:87602:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787800/0xef30c340bfb4703c lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b832637e expref: -99 pid: 85409 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:2.0:1788572249.252400:0:87602:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787800/0xef30c340bfb4703c lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b832637e expref: -99 pid: 85409 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:2.0:1788572249.252410:0:87602:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788572249.252417:0:87602:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788572249.252421:0:87602:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4703c 00000080:00010000:2.0:1788572249.252430:0:87602:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4703c from it ffffab0ec0dabbf8 00010000:00010000:2.0:1788572249.252434:0:87602:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787800/0xef30c340bfb4703c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b832637e expref: -99 pid: 85409 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:2.0:1788572249.252442:0:87602:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787800/0xef30c340bfb4703c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b832637e expref: -99 pid: 85409 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000080:00200000:2.0:1788572249.252455:0:87602:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:2.0:1788572249.252470:0:87602:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788572249.252477:0:87602:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788572249.252484:0:87602:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787800/0xef30c340bfb4703c lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b832637e expref: -99 pid: 85409 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:2.0:1788572249.252492:0:87602:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787800/0xef30c340bfb4703c lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b832637e expref: -99 pid: 85409 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000080:00200000:2.0:1788572249.252498:0:87602:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788572249.252502:0:87602:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788572249.252505:0:87602:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4703c 00000080:00010000:2.0:1788572249.252508:0:87602:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4703c from it ffffab0ec0dabb38 00010000:00010000:2.0:1788572249.252511:0:87602:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787800/0xef30c340bfb4703c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b832637e expref: -99 pid: 85409 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:2.0:1788572249.252524:0:87602:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787800/0xef30c340bfb4703c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b832637e expref: -99 pid: 85409 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00000080:00200000:2.0:1788572249.252531:0:87602:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:2.0:1788572249.252537:0:87602:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f41.replay-single, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), file ffff9ada4a118a80, open_flags a141, mode 81b6 opened 0 00000080:00200000:2.0:1788572249.252546:0:87602:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f41.replay-single, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000002:00010000:2.0:1788572249.252569:0:87602:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f41.replay-single,[0x2000182d1:0x643:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0120502 00010000:00010000:2.0:1788572249.252582:0:87602:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787800/0xef30c340bfb4703c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b832637e expref: -99 pid: 85409 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00010000:00010000:2.0:1788572249.253430:0:87602:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787800/0xef30c340bfb4703c lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b832637e expref: -99 pid: 85409 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00010000:00010000:2.0:1788572249.253441:0:87602:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1788572249.253445:0:87602:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada51787800/0xef30c340bfb4703c lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b832637e expref: -99 pid: 85409 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00010000:00010000:2.0:1788572249.253497:0:87602:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035c600/0xef30c340bfb4704a lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 87602 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572249.253505:0:87602:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035c600/0xef30c340bfb4704a lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 87602 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1788572249.253518:0:87602:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1788572249.253530:0:87602:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada517fc000 x1875449615412352/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 536/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1788572249.253540:0:87602:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035c600/0xef30c340bfb4704a lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 87602 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572249.253564:0:87602:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517fc000 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:87602:1875449615412352:192.168.202.113@tcp:101:lfs.0 00000100:00100000:2.0:1788572249.253737:0:87602:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60478c00 going to sleep for 11 seconds 00000100:00100000:0.0:1788572249.275280:0:87602:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517fc000 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:87602:1875449615412352:192.168.202.113@tcp:101:lfs.0 00010000:00010000:0.0:1788572249.275312:0:87602:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035c600/0xef30c340bfb4704a lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326713 expref: -99 pid: 87602 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572249.275493:0:87602:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035c600/0xef30c340bfb4704a lrc: 4/0,1 mode: --/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326713 expref: -99 pid: 87602 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572249.275504:0:87602:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035c600/0xef30c340bfb4704a lrc: 4/0,1 mode: --/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326713 expref: -99 pid: 87602 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572249.275522:0:87602:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035c600/0xef30c340bfb4704a lrc: 4/0,1 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8326713 expref: -99 pid: 87602 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572249.275535:0:87602:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035c600/0xef30c340bfb4704a lrc: 4/0,1 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326713 expref: -99 pid: 87602 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572249.275551:0:87602:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035c600/0xef30c340bfb4704a lrc: 5/1,1 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8326713 expref: -99 pid: 87602 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572249.275561:0:87602:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035c600/0xef30c340bfb4704a lrc: 5/1,1 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8326713 expref: -99 pid: 87602 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572249.275569:0:87602:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035c600/0xef30c340bfb4704a lrc: 4/1,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8326713 expref: -99 pid: 87602 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1788572249.275583:0:87602:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x8000037, status=0 req@ffff9ada517fc000 x1875449615412352/t219043332497(219043332497) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 536/608 e 0 to 0 dl 1788572265 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1788572249.275607:0:87602:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada517fc000 x1875449615412352/t219043332497(219043332497) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 536/608 e 0 to 0 dl 1788572265 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1788572249.275620:0:87602:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035c600/0xef30c340bfb4704a lrc: 3/1,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326713 expref: -99 pid: 87602 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572249.275632:0:87602:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8590033617/1603 (0 0) 00000080:00200000:1.0:1788572249.275639:0:87602:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1788572249.275653:0:87602:0:(namei.c:1628:ll_create_it()) VFS Op:name=f41.replay-single, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000080:00200000:1.0:1788572249.275691:0:87602:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x2000182d1:0x643:0x0](ffff9ada58237448) 00000080:00200000:1.0:1788572249.275698:0:87602:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x2000182d1:0x643:0x0], updating i_size 0 00000080:00200000:1.0:1788572249.275737:0:87602:0:(file.c:6843:ll_layout_conf()) [0x2000182d1:0x643:0x0]: layout version change: 4294967294 -> 4294967295 00000080:00010000:1.0:1788572249.275742:0:87602:0:(namei.c:1594:ll_create_node()) setting l_ast_data to inode [0x2000182d1:0x643:0x0](ffff9ada58237448) 00000080:00010000:1.0:1788572249.275746:0:87602:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x2000182d1:0x643:0x0](ffff9ada58237448) for lock 0xef30c340bfb4704a 00000080:00010000:1.0:1788572249.275756:0:87602:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x2000182d1:0x643:0x0](ffff9ada58237448) 00000080:00200008:1.0:1788572249.275769:0:87602:0:(file.c:1090:ll_file_open()) START file f41.replay-single:[0x2000182d1:0x643:0x0](ffff9ada58237448), flags 120501 00000002:00100000:1.0:1788572249.275790:0:87602:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada517fc000 x1875449615412352/t219043332497(219043332497) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 536/608 e 0 to 0 dl 1788572265 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00010000:1.0:1788572249.275824:0:87602:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4704a from it ffff9ada60781960 00010000:00010000:1.0:1788572249.275828:0:87602:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035c600/0xef30c340bfb4704a lrc: 3/1,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b8326713 expref: -99 pid: 87602 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572249.275838:0:87602:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035c600/0xef30c340bfb4704a lrc: 2/0,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b8326713 expref: -99 pid: 87602 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572249.275894:0:87602:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x2000182d1:0x643:0x0](ffff9ada58237448), inode mode 81a4 mask 40 00000080:00200080:1.0:1788572249.275925:0:87602:0:(file.c:4646:ll_file_ioctl()) VFS Op:inode=[0x2000182d1:0x643:0x0](ffff9ada58237448) cmd=4008669a arg=24c0530 00000002:00010000:1.0:1788572249.275948:0:87602:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x2000182d1:0x643:0x0]) in obj [0x200000007:0x1:0x0], intent: open flags 040000002 00010000:00010000:1.0:1788572249.276000:0:87602:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4542a400/0xef30c340bfb47051 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 87602 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572249.276008:0:87602:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4542a400/0xef30c340bfb47051 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 87602 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:1.0:1788572249.276020:0:87602:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1788572249.276027:0:87602:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada45413b80 x1875449615412608/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 648/131856 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1788572249.276041:0:87602:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4542a400/0xef30c340bfb47051 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 87602 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572249.276064:0:87602:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45413b80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:87602:1875449615412608:192.168.202.113@tcp:101:lfs.0 00000100:00100000:1.0:1788572249.276295:0:87602:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60461cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572249.291118:0:2378:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.113@tcp (source: 12345-192.168.202.113@tcp) 00000100:00100000:3.0:1788572249.291206:0:15196:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:3.0:1788572249.291228:0:15196:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875449639004288 00000100:00100000:3.0:1788572249.291238:0:15196:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875449639004288 00000100:00100000:3.0:1788572249.291298:0:15196:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.113@tcp, seq: 237 00000100:00100000:3.0:1788572249.291309:0:15196:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9ada50b6b850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_000:35eb2d60-2de0-4b00-9815-d5153edd0057+5:6551:x1875449639004288:12345-192.168.202.113@tcp:104: 00010000:00010000:3.0:1788572249.291328:0:15196:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035c600/0xef30c340bfb4704a lrc: 2/0,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x420000000000 nid: local remote: 0xdb7edb52b8326713 expref: -99 pid: 87602 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1788572249.291417:0:15196:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1788572249.291426:0:15196:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9ada50b6b850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_000:35eb2d60-2de0-4b00-9815-d5153edd0057+5:6551:x1875449639004288:12345-192.168.202.113@tcp:104: Request processed in 118us (329us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572249.291433:0:15196:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.113@tcp, seq: 237 00010000:00010000:0.0:1788572249.291721:0:15205:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1788572249.291730:0:15205:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035c600/0xef30c340bfb4704a lrc: 2/0,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x420000000000 nid: local remote: 0xdb7edb52b8326713 expref: -99 pid: 87602 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788572249.291739:0:15205:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada6035c600 already unused, calling callback (ffffffffc1966630) 00010000:00010000:0.0:1788572249.291745:0:15205:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035c600/0xef30c340bfb4704a lrc: 2/0,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x1b rrc: 2 type: IBT gid 0 flags: 0x420400000000 nid: local remote: 0xdb7edb52b8326713 expref: -99 pid: 87602 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788572249.291754:0:15205:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035c600/0xef30c340bfb4704a lrc: 2/0,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x1b rrc: 2 type: IBT gid 0 flags: 0x420400000000 nid: local remote: 0xdb7edb52b8326713 expref: -99 pid: 87602 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788572249.291763:0:15205:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035c600/0xef30c340bfb4704a lrc: 3/0,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x1b rrc: 2 type: IBT gid 0 flags: 0x428400000000 nid: local remote: 0xdb7edb52b8326713 expref: -99 pid: 87602 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572249.291788:0:15205:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x643:0x0]: losing layout lock 00010000:00010000:0.0:1788572249.291833:0:15205:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035c600/0xef30c340bfb4704a lrc: 2/0,0 mode: --/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x1b rrc: 2 type: IBT gid 0 flags: 0x4c29400000020 nid: local remote: 0xdb7edb52b8326713 expref: -99 pid: 87602 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788572249.291841:0:15205:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1788572249.291901:0:15205:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035c600/0xef30c340bfb4704a lrc: 1/0,0 mode: --/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x1b rrc: 2 type: IBT gid 0 flags: 0x4c29400000020 nid: local remote: 0xdb7edb52b8326713 expref: -99 pid: 87602 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788572249.291912:0:15205:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035c600/0xef30c340bfb4704a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x1b rrc: 2 type: IBT gid 0 flags: 0x4c29400000020 nid: local remote: 0xdb7edb52b8326713 expref: -99 pid: 87602 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788572249.291935:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572249.291937:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572249.291939:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572249.291957:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a34700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615412736:192.168.202.113@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1788572249.294685:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a34700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615412736:192.168.202.113@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1788572249.295625:0:87602:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45413b80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:87602:1875449615412608:192.168.202.113@tcp:101:lfs.0 00010000:00010000:1.0:1788572249.295646:0:87602:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4542a400/0xef30c340bfb47051 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 87602 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572249.295661:0:87602:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4542a400/0xef30c340bfb47051 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 87602 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572249.295668:0:87602:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4542a400/0xef30c340bfb47051 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 87602 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572249.295675:0:87602:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4542a400/0xef30c340bfb47051 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 87602 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572249.295684:0:87602:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4542a400/0xef30c340bfb47051 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 87602 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572249.295690:0:87602:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada4542a400 already unused, calling callback (ffffffffc1966630) 00010000:00010000:1.0:1788572249.295695:0:87602:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4542a400/0xef30c340bfb47051 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 2 type: IBT flags: 0x206400000000 pid: 87602 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572249.295702:0:87602:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4542a400/0xef30c340bfb47051 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 2 type: IBT flags: 0x206400000000 pid: 87602 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572249.295709:0:87602:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4542a400/0xef30c340bfb47051 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 2 type: IBT flags: 0x20e400000000 pid: 87602 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572249.295730:0:87602:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1788572249.295737:0:87602:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4542a400/0xef30c340bfb47051 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 87602 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572249.295745:0:87602:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada4542a400/0xef30c340bfb47051 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 87602 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1788572249.295774:0:87602:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x1 disposition=0x800002b, status=0 req@ffff9ada45413b80 x1875449615412608/t219043332498(219043332498) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 648/664 e 0 to 0 dl 1788572265 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1788572249.295788:0:87602:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada45413b80 x1875449615412608/t219043332498(219043332498) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 648/664 e 0 to 0 dl 1788572265 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1788572249.295804:0:87602:0:(namei.c:952:ll_intent_lock()) intent lock 1 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1788572249.295884:0:87602:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x643:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1788572249.295895:0:87602:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x643:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1788572249.295944:0:87602:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x2000182d1:0x643:0x0], updating i_size 0 00000002:00100000:1.0:1788572249.295961:0:87602:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada45413b80 x1875449615412608/t219043332498(219043332498) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/664 e 0 to 0 dl 1788572265 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1788572249.295998:0:87602:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9ada45413b80 x1875449615412608/t219043332498(219043332498) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/664 e 0 to 0 dl 1788572265 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000020:00100000:1.0:1788572249.296007:0:87602:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:1.0:1788572249.296013:0:87602:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada45410000 x1875449615412992/t0(0) o35->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:1.0:1788572249.296035:0:87602:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45410000 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:87602:1875449615412992:192.168.202.113@tcp:35:lfs.0 00000100:00100000:1.0:1788572249.296110:0:87602:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60461180 going to sleep for 11 seconds 00000100:00100000:0.0:1788572249.303432:0:87602:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45410000 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:87602:1875449615412992:192.168.202.113@tcp:35:lfs.0 00000002:00100000:0.0:1788572249.303475:0:87602:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9ada45413b80 x1875449615412608/t219043332498(219043332498) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/664 e 0 to 0 dl 1788572265 ref 2 fl Complete:RQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00010000:0.0:1788572249.303513:0:87602:0:(file.c:7065:ll_layout_intent()) ### lustre: requeue layout lock for file [0x2000182d1:0x643:0x0](ffff9ada58237448) 00000002:00010000:0.0:1788572249.303526:0:87602:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x2000182d1:0x643:0x0]) in obj [0x2000182d1:0x643:0x0], intent: layout flags 0 00010000:00010000:0.0:1788572249.303584:0:87602:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9600/0xef30c340bfb47058 lrc: 3/1,0 mode: --/CR res: [0x2000182d1:0x643:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 87602 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572249.303596:0:87602:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9600/0xef30c340bfb47058 lrc: 3/1,0 mode: --/CR res: [0x2000182d1:0x643:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x0 pid: 87602 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572249.303607:0:87602:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9600/0xef30c340bfb47058 lrc: 3/1,0 mode: --/CR res: [0x2000182d1:0x643:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x0 pid: 87602 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572249.303630:0:87602:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a34700 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:87602:1875449615413120:192.168.202.113@tcp:101:lfs.0 00000100:00100000:0.0:1788572249.303722:0:87602:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada519bbcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572249.312438:0:87602:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a34700 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:87602:1875449615413120:192.168.202.113@tcp:101:lfs.0 00010000:00010000:0.0:1788572249.312480:0:87602:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9600/0xef30c340bfb47058 lrc: 4/1,0 mode: CR/CR res: [0x2000182d1:0x643:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8326736 expref: -99 pid: 87602 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572249.312496:0:87602:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9600/0xef30c340bfb47058 lrc: 4/1,0 mode: CR/CR res: [0x2000182d1:0x643:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326736 expref: -99 pid: 87602 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788572249.312512:0:87602:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x400 disposition=0x1, status=0 req@ffff9ada60a34700 x1875449615413120/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 376/400 e 0 to 0 dl 1788572265 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788572249.312525:0:87602:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: layout, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9600/0xef30c340bfb47058 lrc: 3/1,0 mode: CR/CR res: [0x2000182d1:0x643:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326736 expref: -99 pid: 87602 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00000002:00010000:0.0:1788572249.312539:0:87602:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9600/0xef30c340bfb47058 lrc: 3/1,0 mode: CR/CR res: [0x2000182d1:0x643:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326736 expref: -99 pid: 87602 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572249.312553:0:87602:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 16 res 8590033617/1603 (0 0) 00000080:00200000:0.0:1788572249.312559:0:87602:0:(namei.c:952:ll_intent_lock()) intent lock 1024 on i1 [0x2000182d1:0x643:0x0] suppgids 0 0: rc 0 00000080:00010000:0.0:1788572249.312577:0:87602:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x2000182d1:0x643:0x0](ffff9ada58237448) for lock 0xef30c340bfb47058 00000080:00010000:0.0:1788572249.312587:0:87602:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb47058 from it ffffab0ec0dabc38 00010000:00010000:0.0:1788572249.312591:0:87602:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9600/0xef30c340bfb47058 lrc: 3/1,0 mode: CR/CR res: [0x2000182d1:0x643:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8326736 expref: -99 pid: 87602 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572249.312603:0:87602:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9600/0xef30c340bfb47058 lrc: 2/0,0 mode: CR/CR res: [0x2000182d1:0x643:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326736 expref: -99 pid: 87602 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788572249.312630:0:87602:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9600/0xef30c340bfb47058 lrc: 2/1,0 mode: CR/CR res: [0x2000182d1:0x643:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326736 expref: -99 pid: 87602 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788572249.312641:0:87602:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9600/0xef30c340bfb47058 lrc: 2/1,0 mode: CR/CR res: [0x2000182d1:0x643:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326736 expref: -99 pid: 87602 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00010000:0.0:1788572249.312650:0:87602:0:(file.c:6948:ll_layout_lock_set()) ### file [0x2000182d1:0x643:0x0](00000000bf070a72) being reconfigured ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9600/0xef30c340bfb47058 lrc: 3/1,0 mode: CR/CR res: [0x2000182d1:0x643:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326736 expref: -99 pid: 87602 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572249.312699:0:87602:0:(file.c:6843:ll_layout_conf()) [0x2000182d1:0x643:0x0]: layout version change: 4294967294 -> 0 00010000:00010000:0.0:1788572249.312706:0:87602:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9600/0xef30c340bfb47058 lrc: 3/1,0 mode: CR/CR res: [0x2000182d1:0x643:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b8326736 expref: -99 pid: 87602 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788572249.312714:0:87602:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9600/0xef30c340bfb47058 lrc: 2/0,0 mode: CR/CR res: [0x2000182d1:0x643:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b8326736 expref: -99 pid: 87602 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00000080:00200008:0.0:1788572249.312769:0:87602:0:(file.c:479:ll_file_release()) START file f41.replay-single:[0x2000182d1:0x643:0x0](ffff9ada58237448), flags 100001 00000002:00100000:0.0:1788572249.312816:0:87602:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9ada517fc000 x1875449615412352/t219043332497(219043332497) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 536/608 e 0 to 0 dl 1788572265 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000020:00100000:0.0:1788572249.312826:0:87602:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:0.0:1788572249.312832:0:87602:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada60a34700 x1875449615413376/t0(0) o35->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:0.0:1788572249.319164:0:87602:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a34700 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:87602:1875449615413376:192.168.202.113@tcp:35:lfs.0 00000100:00100000:0.0:1788572249.319285:0:87602:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada519bbcc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788572249.324658:0:87602:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a34700 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:87602:1875449615413376:192.168.202.113@tcp:35:lfs.0 00000002:00100000:1.0:1788572249.324698:0:87602:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9ada517fc000 x1875449615412352/t219043332497(219043332497) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 536/608 e 0 to 0 dl 1788572265 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1788572249.427076:0:87616:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788572249.427122:0:87616:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788572249.428848:0:87616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035d800/0xef30c340bfb4705f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 87616 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572249.428894:0:87616:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035d800/0xef30c340bfb4705f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 87616 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572249.428905:0:87616:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035d800/0xef30c340bfb4705f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 87616 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572249.428936:0:87616:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517fd180 pname:cluuid:pid:xid:nid:opc:job dd:35eb2d60-2de0-4b00-9815-d5153edd0057:87616:1875449615413504:192.168.202.113@tcp:101:dd.0 00000100:00100000:2.0:1788572249.429030:0:87616:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60478cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572249.448287:0:87616:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517fd180 pname:cluuid:pid:xid:nid:opc:job dd:35eb2d60-2de0-4b00-9815-d5153edd0057:87616:1875449615413504:192.168.202.113@tcp:101:dd.0 00010000:00010000:0.0:1788572249.448324:0:87616:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035d800/0xef30c340bfb4705f lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326744 expref: -99 pid: 87616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572249.448337:0:87616:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035d800/0xef30c340bfb4705f lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326744 expref: -99 pid: 87616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572249.448355:0:87616:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035d800/0xef30c340bfb4705f lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8326744 expref: -99 pid: 87616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572249.448367:0:87616:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035d800/0xef30c340bfb4705f lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326744 expref: -99 pid: 87616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572249.448381:0:87616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035d800/0xef30c340bfb4705f lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8326744 expref: -99 pid: 87616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572249.448390:0:87616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035d800/0xef30c340bfb4705f lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8326744 expref: -99 pid: 87616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572249.448398:0:87616:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035d800/0xef30c340bfb4705f lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8326744 expref: -99 pid: 87616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788572249.448412:0:87616:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada517fd180 x1875449615413504/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788572265 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788572249.448426:0:87616:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035d800/0xef30c340bfb4705f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326744 expref: -99 pid: 87616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572249.448439:0:87616:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1788572249.448446:0:87616:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1788572249.448474:0:87616:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:0.0:1788572249.448480:0:87616:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4705f 00000080:00010000:0.0:1788572249.448486:0:87616:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572249.448490:0:87616:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4705f from it ffffab0ec1463bf8 00010000:00010000:0.0:1788572249.448494:0:87616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035d800/0xef30c340bfb4705f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8326744 expref: -99 pid: 87616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572249.448503:0:87616:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035d800/0xef30c340bfb4705f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326744 expref: -99 pid: 87616 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572249.448529:0:87616:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:0.0:1788572249.448559:0:87616:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572249.448576:0:87616:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572249.448586:0:87616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035d800/0xef30c340bfb4705f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326744 expref: -99 pid: 87616 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788572249.454066:0:87616:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035d800/0xef30c340bfb4705f lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326744 expref: -99 pid: 87616 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572249.454085:0:87616:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572249.454104:0:87616:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572249.454109:0:87616:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4705f 00000080:00010000:0.0:1788572249.454119:0:87616:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4705f from it ffffab0ec1463b38 00010000:00010000:0.0:1788572249.454123:0:87616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035d800/0xef30c340bfb4705f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8326744 expref: -99 pid: 87616 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788572249.454134:0:87616:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035d800/0xef30c340bfb4705f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326744 expref: -99 pid: 87616 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788572249.454153:0:87616:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:0.0:1788572249.454162:0:87616:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f41.replay-single, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), file ffff9ada50407080, open_flags 8241, mode 81b6 opened 0 00000080:00200000:0.0:1788572249.454174:0:87616:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f41.replay-single, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open|creat 00000002:00010000:0.0:1788572249.454205:0:87616:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f41.replay-single,[0x2000182d1:0x644:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0101102 00010000:00010000:0.0:1788572249.454222:0:87616:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035d800/0xef30c340bfb4705f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b8326744 expref: -99 pid: 87616 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788572249.454277:0:87616:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035d800/0xef30c340bfb4705f lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b8326744 expref: -99 pid: 87616 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788572249.454284:0:87616:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1788572249.454287:0:87616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035d800/0xef30c340bfb4705f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b8326744 expref: -99 pid: 87616 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788572249.454324:0:87616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9200/0xef30c340bfb47066 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 87616 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572249.454331:0:87616:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9200/0xef30c340bfb47066 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 87616 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:0.0:1788572249.454343:0:87616:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:0.0:1788572249.454355:0:87616:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada60a35500 x1875449615413760/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/66320 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:0.0:1788572249.454365:0:87616:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9200/0xef30c340bfb47066 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 87616 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572249.454387:0:87616:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35500 pname:cluuid:pid:xid:nid:opc:job dd:35eb2d60-2de0-4b00-9815-d5153edd0057:87616:1875449615413760:192.168.202.113@tcp:101:dd.0 00000100:00100000:0.0:1788572249.454562:0:87616:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada519bbb40 going to sleep for 11 seconds 00000100:00100000:0.0:1788572249.467134:0:87616:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a35500 pname:cluuid:pid:xid:nid:opc:job dd:35eb2d60-2de0-4b00-9815-d5153edd0057:87616:1875449615413760:192.168.202.113@tcp:101:dd.0 00010000:00010000:0.0:1788572249.467168:0:87616:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9200/0xef30c340bfb47066 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326760 expref: -99 pid: 87616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572249.467200:0:87616:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9200/0xef30c340bfb47066 lrc: 4/0,1 mode: --/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326760 expref: -99 pid: 87616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572249.467209:0:87616:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9200/0xef30c340bfb47066 lrc: 4/0,1 mode: --/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326760 expref: -99 pid: 87616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572249.467222:0:87616:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9200/0xef30c340bfb47066 lrc: 4/0,1 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8326760 expref: -99 pid: 87616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572249.467232:0:87616:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9200/0xef30c340bfb47066 lrc: 4/0,1 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326760 expref: -99 pid: 87616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572249.467247:0:87616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9200/0xef30c340bfb47066 lrc: 5/1,1 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8326760 expref: -99 pid: 87616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572249.467254:0:87616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9200/0xef30c340bfb47066 lrc: 5/1,1 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8326760 expref: -99 pid: 87616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572249.467262:0:87616:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9200/0xef30c340bfb47066 lrc: 4/1,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8326760 expref: -99 pid: 87616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788572249.467274:0:87616:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x3 disposition=0x2b, status=0 req@ffff9ada60a35500 x1875449615413760/t219043332501(219043332501) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/664 e 0 to 0 dl 1788572265 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00000002:00100000:0.0:1788572249.467287:0:87616:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada60a35500 x1875449615413760/t219043332501(219043332501) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/664 e 0 to 0 dl 1788572265 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788572249.467300:0:87616:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9200/0xef30c340bfb47066 lrc: 3/1,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326760 expref: -99 pid: 87616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:0.0:1788572249.467316:0:87616:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9200/0xef30c340bfb47066 lrc: 3/1,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326760 expref: -99 pid: 87616 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572249.467327:0:87616:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8590033617/1603 (0 0) 00000080:00200000:0.0:1788572249.467334:0:87616:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1788572249.467381:0:87616:0:(namei.c:147:ll_iget()) got inode: [0x2000182d1:0x643:0x0](ffff9ada58237448): rc = 0 00000080:00200000:0.0:1788572249.467395:0:87616:0:(file.c:6843:ll_layout_conf()) [0x2000182d1:0x643:0x0]: layout version change: 0 -> 0 00000080:00010000:0.0:1788572249.467401:0:87616:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x2000182d1:0x643:0x0](ffff9ada58237448) for lock 0xef30c340bfb47066 00000080:00010000:0.0:1788572249.467413:0:87616:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x2000182d1:0x643:0x0](ffff9ada58237448) 00000080:00200008:0.0:1788572249.467429:0:87616:0:(file.c:1090:ll_file_open()) START file f41.replay-single:[0x2000182d1:0x643:0x0](ffff9ada58237448), flags 101101 00000002:00100000:0.0:1788572249.467454:0:87616:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada60a35500 x1875449615413760/t219043332501(219043332501) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788572265 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00000080:00010000:0.0:1788572249.467488:0:87616:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb47066 from it ffff9ada51721960 00010000:00010000:0.0:1788572249.467493:0:87616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9200/0xef30c340bfb47066 lrc: 3/1,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b8326760 expref: -99 pid: 87616 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572249.467503:0:87616:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9200/0xef30c340bfb47066 lrc: 2/0,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b8326760 expref: -99 pid: 87616 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572249.467524:0:87616:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x2000182d1:0x643:0x0](ffff9ada58237448), inode mode 81a4 mask 42 00000002:00010000:0.0:1788572249.467553:0:87616:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x0:0x0:0x0]) in obj [0x2000182d1:0x643:0x0], intent: getxattr flags 0 00010000:00010000:0.0:1788572249.467593:0:87616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9400/0xef30c340bfb4706d lrc: 3/1,0 mode: --/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 87616 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572249.467610:0:87616:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9400/0xef30c340bfb4706d lrc: 3/1,0 mode: --/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x0 pid: 87616 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572249.467622:0:87616:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9400/0xef30c340bfb4706d lrc: 3/1,0 mode: --/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x0 pid: 87616 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572249.467638:0:87616:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a34380 pname:cluuid:pid:xid:nid:opc:job dd:35eb2d60-2de0-4b00-9815-d5153edd0057:87616:1875449615413888:192.168.202.113@tcp:101:dd.0 00000100:00100000:0.0:1788572249.467704:0:87616:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada519bbb40 going to sleep for 11 seconds 00000100:00100000:0.0:1788572249.471815:0:87616:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a34380 pname:cluuid:pid:xid:nid:opc:job dd:35eb2d60-2de0-4b00-9815-d5153edd0057:87616:1875449615413888:192.168.202.113@tcp:101:dd.0 00010000:00010000:0.0:1788572249.471846:0:87616:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9400/0xef30c340bfb4706d lrc: 4/1,0 mode: --/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326775 expref: -99 pid: 87616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572249.471882:0:87616:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9400/0xef30c340bfb4706d lrc: 4/1,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8326775 expref: -99 pid: 87616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572249.471894:0:87616:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9400/0xef30c340bfb4706d lrc: 4/1,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326775 expref: -99 pid: 87616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788572249.471911:0:87616:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x80 disposition=0x0, status=0 req@ffff9ada60a34380 x1875449615413888/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 568/1152 e 0 to 0 dl 1788572265 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788572249.471923:0:87616:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9400/0xef30c340bfb4706d lrc: 3/1,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326775 expref: -99 pid: 87616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572249.471938:0:87616:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8590033617/1603 (0 0) 00000080:00200000:0.0:1788572249.471944:0:87616:0:(namei.c:952:ll_intent_lock()) intent lock 128 on i1 [0x2000182d1:0x643:0x0] suppgids 0 -1: rc 0 00000080:00010000:0.0:1788572249.471986:0:87616:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x2000182d1:0x643:0x0](ffff9ada58237448) for lock 0xef30c340bfb4706d 00000080:00010000:0.0:1788572249.471996:0:87616:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4706d from it ffffab0ec1463a38 00010000:00010000:0.0:1788572249.472000:0:87616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9400/0xef30c340bfb4706d lrc: 3/1,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8326775 expref: -99 pid: 87616 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572249.472017:0:87616:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9400/0xef30c340bfb4706d lrc: 2/0,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326775 expref: -99 pid: 87616 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:0.0:1788572249.472066:0:87616:0:(llite_lib.c:2427:ll_setattr_raw()) START file f41.replay-single:[0x2000182d1:0x643:0x0](ffff9ada58237448) current size 0, valid attrs a068, mode 0, uid 2, gid 0, new size 0, atime 1788572249.000000000, mtime 1788572249.000000000, ctime 1788572249.000000000 00010000:00010000:0.0:1788572249.472106:0:87616:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9200/0xef30c340bfb47066 lrc: 2/0,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0xdb7edb52b8326760 expref: -99 pid: 87616 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788572249.472127:0:87616:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9600/0xef30c340bfb47058 lrc: 2/0,0 mode: CR/CR res: [0x2000182d1:0x643:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b8326736 expref: -99 pid: 87602 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1788572249.472165:0:87616:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9200/0xef30c340bfb47066 lrc: 1/0,0 mode: --/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0xdb7edb52b8326760 expref: -99 pid: 87616 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788572249.472172:0:87616:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1788572249.472175:0:87616:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9200/0xef30c340bfb47066 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0xdb7edb52b8326760 expref: -99 pid: 87616 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1788572249.472192:0:87616:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1788572249.472199:0:87616:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada60a34380 x1875449615414144/t0(0) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 488/512 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788572249.472222:0:87616:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a34380 pname:cluuid:pid:xid:nid:opc:job dd:35eb2d60-2de0-4b00-9815-d5153edd0057:87616:1875449615414144:192.168.202.113@tcp:36:dd.0 00000100:00100000:0.0:1788572249.472300:0:87616:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada519bbb40 going to sleep for 11 seconds 00000100:00100000:0.0:1788572249.481224:0:87616:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a34380 pname:cluuid:pid:xid:nid:opc:job dd:35eb2d60-2de0-4b00-9815-d5153edd0057:87616:1875449615414144:192.168.202.113@tcp:36:dd.0 00000080:00200000:0.0:1788572249.481279:0:87616:0:(file.c:1848:ll_io_set_mirror()) f41.replay-single: desiginated mirror: 0 00000080:00200000:0.0:1788572249.481287:0:87616:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x643:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00020000:00200000:0.0:1788572249.481307:0:87616:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1788572249.481322:0:87616:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [0, 18446744073709551615] 00000008:00010000:0.0:1788572249.481339:0:87616:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 4 00000008:00010000:0.0:1788572249.481356:0:87616:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff9ada60a53cb0, osc lock ffff9ada7d07abe0, flags 0x80000000 00010000:00010000:0.0:1788572249.481417:0:87616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000b753e25c/0xef30c340bfb47074 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 87616 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572249.481431:0:87616:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x80000000 ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000b753e25c/0xef30c340bfb47074 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 87616 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572249.481442:0:87616:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000b753e25c/0xef30c340bfb47074 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 87616 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572249.481464:0:87616:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a34e00 pname:cluuid:pid:xid:nid:opc:job dd:35eb2d60-2de0-4b00-9815-d5153edd0057:87616:1875449615414272:192.168.202.113@tcp:101:dd.0 00000100:00100000:0.0:1788572249.481548:0:87616:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada519bbb40 going to sleep for 11 seconds 00000100:00100000:0.0:1788572249.486353:0:87616:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a34e00 pname:cluuid:pid:xid:nid:opc:job dd:35eb2d60-2de0-4b00-9815-d5153edd0057:87616:1875449615414272:192.168.202.113@tcp:101:dd.0 00010000:00010000:0.0:1788572249.486390:0:87616:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000b753e25c/0xef30c340bfb47074 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000080000000 nid: local remote: 0xdb7edb52b8326783 expref: -99 pid: 87616 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572249.486408:0:87616:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000b753e25c/0xef30c340bfb47074 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x80000000 nid: local remote: 0xdb7edb52b8326783 expref: -99 pid: 87616 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572249.486425:0:87616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000b753e25c/0xef30c340bfb47074 lrc: 5/0,2 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000080000000 nid: local remote: 0xdb7edb52b8326783 expref: -99 pid: 87616 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1788572249.486438:0:87616:0:(osc_lock.c:187:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000b753e25c/0xef30c340bfb47074 lrc: 4/0,2 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020080000000 nid: local remote: 0xdb7edb52b8326783 expref: -99 pid: 87616 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572249.486452:0:87616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000b753e25c/0xef30c340bfb47074 lrc: 5/0,2 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020080000000 nid: local remote: 0xdb7edb52b8326783 expref: -99 pid: 87616 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572249.486461:0:87616:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000b753e25c/0xef30c340bfb47074 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020080000000 nid: local remote: 0xdb7edb52b8326783 expref: -99 pid: 87616 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572249.486567:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1788572249.486589:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a34e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615414400:192.168.202.113@tcp:10:dd.0 00000100:00100000:0.0:1788572249.492913:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a34e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615414400:192.168.202.113@tcp:10:dd.0 00000080:00200000:2.0:1788572249.493045:0:87616:0:(file.c:1775:ll_merge_attr_nolock()) [0x2000182d1:0x643:0x0] updating i_size 0 i_blocks 0 00000020:00010000:2.0:1788572249.493053:0:87616:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9ada5169a998 (ffff9ada60b0d1b8/0) at cl_lock_release():245 00000020:00010000:2.0:1788572249.493057:0:87616:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9ada5169a998 (ffff9ada60b0d1b8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1788572249.493062:0:87616:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9ada60a53cb0 (ffff9ada60b0d998/1) at cl_lock_cancel():142 00010000:00010000:2.0:1788572249.493073:0:87616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000b753e25c/0xef30c340bfb47074 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020080000000 nid: local remote: 0xdb7edb52b8326783 expref: -99 pid: 87616 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572249.493088:0:87616:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000b753e25c/0xef30c340bfb47074 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020080000000 nid: local remote: 0xdb7edb52b8326783 expref: -99 pid: 87616 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00010000:2.0:1788572249.493113:0:87616:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9ada5169a998 (ffff9ada60b0d1b8/0) at cl_lock_fini():72 00000020:00010000:2.0:1788572249.493118:0:87616:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9ada60a53cb0 (ffff9ada60b0d1b8/1) at cl_lock_fini():72 00000080:00200000:2.0:1788572249.493158:0:87616:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x643:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200008:2.0:1788572249.493183:0:87616:0:(llite_lib.c:2686:ll_setattr_raw()) COMPLETED file f41.replay-single:[0x2000182d1:0x643:0x0](ffff9ada58237448) current size 0, valid attrs 1000a168, mode 0, uid 2, gid 0, new size 0, atime 1788572249.000000000, mtime 1788572249.000000000, ctime 1788572249.000000000, rc 0 00000080:00200008:2.0:1788572249.493298:0:87616:0:(file.c:2650:do_file_write_iter()) START file f41.replay-single:[0x2000182d1:0x643:0x0], ppos: 0, count: 4096 00000080:00200000:2.0:1788572249.493308:0:87616:0:(file.c:2090:ll_file_io_generic()) f41.replay-single: write ppos: 0, bytes: 4096 00000080:00200000:2.0:1788572249.493314:0:87616:0:(file.c:1848:ll_io_set_mirror()) f41.replay-single: desiginated mirror: 0 00000020:00200000:2.0:1788572249.493320:0:87616:0:(cl_io.c:214:cl_io_rw_init()) header@00000000cb95b7e8[0x110, 1, [0x2000182d1:0x643:0x0] hash] 00000020:00200000:2.0:1788572249.493323:0:87616:0:(cl_io.c:214:cl_io_rw_init()) io range: 2 [0, 4096) 0 0 00000080:00200000:2.0:1788572249.493328:0:87616:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x643:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00200000:2.0:1788572249.493342:0:87616:0:(file.c:2168:ll_file_io_generic()) Range lock [0, 0] 00020000:00200000:2.0:1788572249.493349:0:87616:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 0 chunk: [0, 4096) 4096, 4096 00020000:00200000:2.0:1788572249.493353:0:87616:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1788572249.493369:0:87616:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 1 [0, 0] 00000008:00010000:2.0:1788572249.493383:0:87616:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 0 00000008:00010000:2.0:1788572249.493388:0:87616:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff9ada60704730, osc lock ffff9ada4880e720, flags 0x0 00010000:00010000:2.0:1788572249.493408:0:87616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000b753e25c/0xef30c340bfb47074 lrc: 2/0,1 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020080000000 nid: local remote: 0xdb7edb52b8326783 expref: -99 pid: 87616 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788572249.493422:0:87616:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000b753e25c/0xef30c340bfb47074 lrc: 2/0,1 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020080000000 nid: local remote: 0xdb7edb52b8326783 expref: -99 pid: 87616 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788572249.493435:0:87616:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000b753e25c/0xef30c340bfb47074 lrc: 6/0,2 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020080000000 nid: local remote: 0xdb7edb52b8326783 expref: -99 pid: 87616 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788572249.493445:0:87616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000b753e25c/0xef30c340bfb47074 lrc: 6/0,2 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020080000000 nid: local remote: 0xdb7edb52b8326783 expref: -99 pid: 87616 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788572249.493454:0:87616:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000b753e25c/0xef30c340bfb47074 lrc: 5/0,1 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020080000000 nid: local remote: 0xdb7edb52b8326783 expref: -99 pid: 87616 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788572249.493467:0:87616:0:(vvp_io.c:1292:vvp_io_write_start()) f41.replay-single: write [0, 4096) append 0 append lockpos 0 00000080:00200000:2.0:1788572249.493478:0:87616:0:(rw26.c:676:ll_write_begin()) Writing 0 of 0 to 4096 bytes 00000080:00200000:2.0:1788572249.493510:0:87616:0:(rw26.c:871:ll_write_end()) pos 0, len 4096, copied 4096 00000080:00200000:2.0:1788572249.493518:0:87616:0:(rw26.c:929:ll_write_end()) page@0000000044edf6c2[3 000000001b009600 2 1 00000000a56461bc 0 0 0] 00000080:00200000:2.0:1788572249.493522:0:87616:0:(rw26.c:929:ll_write_end()) vmpage @00000000db06b163 17ffffc0001001 3:0 ffff9ada519b8700 0 lru 00000080:00200000:2.0:1788572249.493530:0:87616:0:(rw26.c:929:ll_write_end()) osc-page@00000000dc9060b0 0: 1< 0 - - > 2< 0 0 0 0x0 0x0 | 0000000035172b75 000000000f447a30 > 3< 0 0 > 4< 0 0 8 484638720 - | - - - - > 5< - - - - | 0 - | 0 - -> 00000080:00200000:2.0:1788572249.493533:0:87616:0:(rw26.c:929:ll_write_end()) end page@0000000044edf6c2 00000080:00200000:2.0:1788572249.493534:0:87616:0:(rw26.c:929:ll_write_end()) queued cl_page: 1. 00000080:00200000:2.0:1788572249.493539:0:87616:0:(vvp_io.c:1175:vvp_io_write_commit()) commit async pages: 1, from 0, to 4096 prio 0 00000008:00010000:2.0:1788572249.493559:0:87616:0:(osc_cache.c:858:osc_extent_find()) ### extent: 0000000020e51698 ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000b753e25c/0xef30c340bfb47074 lrc: 4/0,1 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020080000000 nid: local remote: 0xdb7edb52b8326783 expref: -99 pid: 87616 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788572249.493577:0:87616:0:(vvp_io.c:1088:vvp_set_batch_dirty()) mapping ffff9ada582375f8, count 1, dirtied 1 00000080:00200000:2.0:1788572249.493582:0:87616:0:(vvp_io.c:1198:vvp_io_write_commit()) Committed 1 pages 4096 bytes, tot: 4096 00000080:00200000:2.0:1788572249.493585:0:87616:0:(vvp_io.c:1358:vvp_io_write_start()) f41.replay-single: write bytes 0, result: 4096 00000020:00010000:2.0:1788572249.493598:0:87616:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9ada5169a128 (ffff9ada60b0d008/0) at cl_lock_release():245 00000020:00010000:2.0:1788572249.493601:0:87616:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9ada5169a128 (ffff9ada60b0d008/0) at cl_lock_cancel():142 00000020:00010000:2.0:1788572249.493604:0:87616:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9ada60704730 (ffff9ada60b0d248/1) at cl_lock_cancel():142 00010000:00010000:2.0:1788572249.493608:0:87616:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000b753e25c/0xef30c340bfb47074 lrc: 5/0,1 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020080000000 nid: local remote: 0xdb7edb52b8326783 expref: -99 pid: 87616 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788572249.493617:0:87616:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000b753e25c/0xef30c340bfb47074 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020080000000 nid: local remote: 0xdb7edb52b8326783 expref: -99 pid: 87616 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000020:00010000:2.0:1788572249.493626:0:87616:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9ada5169a128 (ffff9ada60b0d008/0) at cl_lock_fini():72 00000020:00010000:2.0:1788572249.493629:0:87616:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9ada60704730 (ffff9ada60b0d008/1) at cl_lock_fini():72 00000080:00200000:2.0:1788572249.493635:0:87616:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000080:00200000:2.0:1788572249.493639:0:87616:0:(file.c:2205:ll_file_io_generic()) Range unlock [0, 0] 00000080:00200000:2.0:1788572249.493644:0:87616:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x643:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1788572249.493648:0:87616:0:(file.c:2238:ll_file_io_generic()) f41.replay-single: 2 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:2.0:1788572249.493653:0:87616:0:(file.c:1775:ll_merge_attr_nolock()) [0x2000182d1:0x643:0x0] updating i_size 4096 i_blocks 0 00000080:00200000:2.0:1788572249.493656:0:87616:0:(file.c:2324:ll_file_io_generic()) iot: 2, result: 4096 00000080:00200008:2.0:1788572249.493681:0:87616:0:(file.c:479:ll_file_release()) START file f41.replay-single:[0x2000182d1:0x643:0x0](ffff9ada58237448), flags 100001 00000002:00100000:2.0:1788572249.493738:0:87616:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9ada60a35500 x1875449615413760/t219043332501(219043332501) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788572265 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1788572249.493750:0:87616:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1788572249.493756:0:87616:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada517fd880 x1875449615414656/t0(0) o35->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1788572249.493777:0:87616:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517fd880 pname:cluuid:pid:xid:nid:opc:job dd:35eb2d60-2de0-4b00-9815-d5153edd0057:87616:1875449615414656:192.168.202.113@tcp:35:dd.0 00000100:00100000:2.0:1788572249.493970:0:87616:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60478f00 going to sleep for 11 seconds 00000100:00100000:0.0:1788572249.499460:0:87616:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517fd880 pname:cluuid:pid:xid:nid:opc:job dd:35eb2d60-2de0-4b00-9815-d5153edd0057:87616:1875449615414656:192.168.202.113@tcp:35:dd.0 00000002:00100000:0.0:1788572249.499502:0:87616:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9ada60a35500 x1875449615413760/t219043332501(219043332501) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788572265 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00010000:00010000:0.0:1788572249.530039:0:87619:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace lustre-OST0000-osc-ffff9ada58b0a000 00010000:00010000:0.0:1788572249.530075:0:87619:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 6/6 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1788572249.530440:0:33423:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 6 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1788572249.530451:0:33423:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 0000000043698fc0/0xef30c340bfb3c747 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x402:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800028480000000 nid: local remote: 0xdb7edb52b82f10d4 expref: -99 pid: 54606 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1788572249.530485:0:33423:0:(vvp_io.c:1826:vvp_io_init()) [0x200001b71:0x12e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:3.0:1788572249.530538:0:33423:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b71:0x12e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:3.0:1788572249.530562:0:33423:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000ec6523cb/0xef30c340bfb3c396 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x302:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800028400000000 nid: local remote: 0xdb7edb52b82ee3e7 expref: -99 pid: 42730 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1788572249.530575:0:33423:0:(vvp_io.c:1826:vvp_io_init()) [0x200001b71:0x118:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:3.0:1788572249.530601:0:33423:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b71:0x118:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:3.0:1788572249.530613:0:33423:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 0000000091ea2998/0xef30c340bfb3a018 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x184:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800028400000000 nid: local remote: 0xdb7edb52b82e1c28 expref: -99 pid: 29118 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1788572249.530626:0:33423:0:(vvp_io.c:1826:vvp_io_init()) [0x200001b71:0x1a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:3.0:1788572249.530648:0:33423:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b71:0x1a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:3.0:1788572249.530660:0:33423:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000b4826d4a/0xef30c340bfb39fee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x183:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800028400000000 nid: local remote: 0xdb7edb52b82e1bb8 expref: -99 pid: 29118 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1788572249.530671:0:33423:0:(vvp_io.c:1826:vvp_io_init()) [0x200001b71:0x18:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:3.0:1788572249.530689:0:33423:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b71:0x18:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:3.0:1788572249.530701:0:33423:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 000000000a521da2/0xef30c340bfb39fc4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x182:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800028400000000 nid: local remote: 0xdb7edb52b82e1b48 expref: -99 pid: 29118 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1788572249.530880:0:87620:0:(ldlm_resource.c:195:lru_size_store()) dropping all unused locks from namespace lustre-OST0001-osc-ffff9ada58b0a000 00010000:00010000:1.0:1788572249.530898:0:87620:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 4/4 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1788572249.530908:0:15204:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 4 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1788572249.530914:0:15204:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 0000000096c928d4/0xef30c340bfb3c357 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x302:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800028480000000 nid: local remote: 0xdb7edb52b82ee2f2 expref: -99 pid: 42730 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1788572249.530931:0:15204:0:(vvp_io.c:1826:vvp_io_init()) [0x200001b71:0x115:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1788572249.530971:0:15204:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b71:0x115:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:1.0:1788572249.530987:0:15204:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 0000000028469bee/0xef30c340bfb3a003 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x184:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800028400000000 nid: local remote: 0xdb7edb52b82e1bf0 expref: -99 pid: 29118 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572249.530998:0:15204:0:(vvp_io.c:1826:vvp_io_init()) [0x200001b71:0x19:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1788572249.531015:0:15204:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b71:0x19:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:1.0:1788572249.531028:0:15204:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 00000000fc9371ec/0xef30c340bfb39fd9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x183:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800028400000000 nid: local remote: 0xdb7edb52b82e1b80 expref: -99 pid: 29118 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572249.531040:0:15204:0:(vvp_io.c:1826:vvp_io_init()) [0x200001b71:0x17:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1788572249.531057:0:15204:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b71:0x17:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:1.0:1788572249.531069:0:15204:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 00000000b52581f9/0xef30c340bfb39faf lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x182:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800028400000000 nid: local remote: 0xdb7edb52b82e1b10 expref: -99 pid: 29118 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572249.531079:0:15204:0:(vvp_io.c:1826:vvp_io_init()) [0x200001b71:0x15:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1788572249.531110:0:15204:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b71:0x15:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:1.0:1788572249.531152:0:15204:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 0000000096c928d4/0xef30c340bfb3c357 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x302:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804869480000020 nid: local remote: 0xdb7edb52b82ee2f2 expref: -99 pid: 42730 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1788572249.531161:0:15204:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 0000000028469bee/0xef30c340bfb3a003 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x184:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x804869400000020 nid: local remote: 0xdb7edb52b82e1bf0 expref: -99 pid: 29118 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1788572249.531169:0:15204:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 00000000fc9371ec/0xef30c340bfb39fd9 lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x183:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x804869400000020 nid: local remote: 0xdb7edb52b82e1b80 expref: -99 pid: 29118 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1788572249.531176:0:15204:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 00000000b52581f9/0xef30c340bfb39faf lrc: 1/0,0 mode: --/PW res: [0x2c0000401:0x182:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x804869400000020 nid: local remote: 0xdb7edb52b82e1b10 expref: -99 pid: 29118 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1788572249.531181:0:15204:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 4 locks packed 00000100:00100000:1.0:1788572249.531205:0:15204:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45410a80 pname:cluuid:pid:xid:nid:opc:job ldlm_bl_01:35eb2d60-2de0-4b00-9815-d5153edd0057:15204:1875449615414784:192.168.202.113@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1788572249.531297:0:15204:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60461300 going to sleep for 11 seconds 00000080:00200000:3.0:1788572249.531814:0:33423:0:(vvp_io.c:1826:vvp_io_init()) [0x200001b71:0x16:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:3.0:1788572249.531840:0:33423:0:(vvp_io.c:304:vvp_io_fini()) [0x200001b71:0x16:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:3.0:1788572249.531878:0:33423:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000b753e25c/0xef30c340bfb47074 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800028480000000 nid: local remote: 0xdb7edb52b8326783 expref: -99 pid: 87616 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:3.0:1788572249.531899:0:33423:0:(osc_cache.c:2074:try_to_add_extent_for_io()) ### extent: 0000000020e51698 ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000b753e25c/0xef30c340bfb47074 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800029480000000 nid: local remote: 0xdb7edb52b8326783 expref: -99 pid: 87616 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:3.0:1788572249.531913:0:33423:0:(osc_cache.c:1174:osc_extent_make_ready()) ### extent: 0000000020e51698 ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000b753e25c/0xef30c340bfb47074 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800029480000000 nid: local remote: 0xdb7edb52b8326783 expref: -99 pid: 87616 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788572249.536625:0:33423:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000b753e25c/0xef30c340bfb47074 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800029480000000 nid: local remote: 0xdb7edb52b8326783 expref: -99 pid: 87616 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00100000:0.0:1788572249.536985:0:33423:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff9ada60a35500 - object 0x280000401:2466 offset 0<>4096 00000100:00100000:1.0:1788572249.537062:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615414912:192.168.202.113@tcp:4:dd.0 00000100:00100000:2.0:1788572249.541681:0:15204:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45410a80 pname:cluuid:pid:xid:nid:opc:job ldlm_bl_01:35eb2d60-2de0-4b00-9815-d5153edd0057:15204:1875449615414784:192.168.202.113@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788572249.541731:0:15204:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 0000000096c928d4/0xef30c340bfb3c357 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x302:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804869480000020 nid: local remote: 0xdb7edb52b82ee2f2 expref: -99 pid: 42730 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:2.0:1788572249.541762:0:15204:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 0000000028469bee/0xef30c340bfb3a003 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x184:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x804869400000020 nid: local remote: 0xdb7edb52b82e1bf0 expref: -99 pid: 29118 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788572249.541776:0:15204:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 00000000fc9371ec/0xef30c340bfb39fd9 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x183:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x804869400000020 nid: local remote: 0xdb7edb52b82e1b80 expref: -99 pid: 29118 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788572249.541788:0:15204:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0001-osc-ffff9ada58b0a000 lock: 00000000b52581f9/0xef30c340bfb39faf lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x182:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x804869400000020 nid: local remote: 0xdb7edb52b82e1b10 expref: -99 pid: 29118 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1788572249.541825:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572249.541828:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572249.541830:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000008:00010000:1.0:1788572249.548063:0:2384:0:(osc_cache.c:908:osc_extent_finish()) ### extent: 0000000020e51698 ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000b753e25c/0xef30c340bfb47074 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800029480000000 nid: local remote: 0xdb7edb52b8326783 expref: -99 pid: 87616 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000008:00010000:1.0:1788572249.548178:0:2384:0:(osc_cache.c:497:__osc_extent_remove()) ### extent: 0000000020e51698 ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000b753e25c/0xef30c340bfb47074 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800029480000000 nid: local remote: 0xdb7edb52b8326783 expref: -99 pid: 87616 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00000100:00100000:1.0:1788572249.548210:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a35500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615414912:192.168.202.113@tcp:4:dd.0 00000080:00200000:0.0:1788572249.548524:0:33423:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x643:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:0.0:1788572249.548575:0:33423:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x643:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00010000:00010000:0.0:1788572249.548628:0:33423:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 0000000043698fc0/0xef30c340bfb3c747 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x402:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804869480000020 nid: local remote: 0xdb7edb52b82f10d4 expref: -99 pid: 54606 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788572249.548646:0:33423:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000ec6523cb/0xef30c340bfb3c396 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x302:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x804869400000020 nid: local remote: 0xdb7edb52b82ee3e7 expref: -99 pid: 42730 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788572249.548656:0:33423:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 0000000091ea2998/0xef30c340bfb3a018 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x184:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x804869400000020 nid: local remote: 0xdb7edb52b82e1c28 expref: -99 pid: 29118 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788572249.548664:0:33423:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000b4826d4a/0xef30c340bfb39fee lrc: 1/0,0 mode: --/PW res: [0x280000401:0x183:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x804869400000020 nid: local remote: 0xdb7edb52b82e1bb8 expref: -99 pid: 29118 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788572249.548673:0:33423:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 000000000a521da2/0xef30c340bfb39fc4 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x182:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x804869400000020 nid: local remote: 0xdb7edb52b82e1b48 expref: -99 pid: 29118 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788572249.548681:0:33423:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000b753e25c/0xef30c340bfb47074 lrc: 1/0,0 mode: --/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804869480000020 nid: local remote: 0xdb7edb52b8326783 expref: -99 pid: 87616 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788572249.548688:0:33423:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 6 locks packed 00000100:00100000:0.0:1788572249.548708:0:33423:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a36a00 pname:cluuid:pid:xid:nid:opc:job ldlm_bl_03:35eb2d60-2de0-4b00-9815-d5153edd0057:33423:1875449615415040:192.168.202.113@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1788572249.549263:0:33423:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada519bbb40 going to sleep for 11 seconds 00000100:00100000:0.0:1788572249.557159:0:33423:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a36a00 pname:cluuid:pid:xid:nid:opc:job ldlm_bl_03:35eb2d60-2de0-4b00-9815-d5153edd0057:33423:1875449615415040:192.168.202.113@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788572249.557191:0:33423:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 0000000043698fc0/0xef30c340bfb3c747 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x402:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804869480000020 nid: local remote: 0xdb7edb52b82f10d4 expref: -99 pid: 54606 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788572249.557218:0:33423:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000ec6523cb/0xef30c340bfb3c396 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x302:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x804869400000020 nid: local remote: 0xdb7edb52b82ee3e7 expref: -99 pid: 42730 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788572249.557238:0:33423:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 0000000091ea2998/0xef30c340bfb3a018 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x184:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x804869400000020 nid: local remote: 0xdb7edb52b82e1c28 expref: -99 pid: 29118 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788572249.557251:0:33423:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000b4826d4a/0xef30c340bfb39fee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x183:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x804869400000020 nid: local remote: 0xdb7edb52b82e1bb8 expref: -99 pid: 29118 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788572249.557262:0:33423:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 000000000a521da2/0xef30c340bfb39fc4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x182:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x804869400000020 nid: local remote: 0xdb7edb52b82e1b48 expref: -99 pid: 29118 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788572249.557274:0:33423:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000b753e25c/0xef30c340bfb47074 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x804869480000020 nid: local remote: 0xdb7edb52b8326783 expref: -99 pid: 87616 timeout: 0 lvb_type: 1 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788572249.557302:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572249.557305:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572249.557307:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572249.975294:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572250.487971:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517fe680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449615415168:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572250.488072:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449615415424:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572250.488113:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517fd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449615415680:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572250.488145:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1788572250.488152:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449615415296:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572250.488174:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449615415552:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572250.494798:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449615415296:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572250.494823:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449615415552:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572250.494835:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517fe680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449615415168:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572250.494847:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449615415424:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572250.494898:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517fd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449615415680:192.168.202.113@tcp:400:kworker.0 00010000:00010000:2.0:1788572250.999985:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572252.024220:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:2.0:1788572252.689216:0:87653:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788572252.689251:0:87653:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788572252.689322:0:87653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035dc00/0xef30c340bfb4707b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 87653 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572252.689335:0:87653:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035dc00/0xef30c340bfb4707b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 87653 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572252.689346:0:87653:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035dc00/0xef30c340bfb4707b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 87653 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572252.689375:0:87653:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517fd500 pname:cluuid:pid:xid:nid:opc:job dd:35eb2d60-2de0-4b00-9815-d5153edd0057:87653:1875449615415808:192.168.202.113@tcp:101:dd.0 00000100:00100000:2.0:1788572252.689529:0:87653:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada604786c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572252.702434:0:87653:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517fd500 pname:cluuid:pid:xid:nid:opc:job dd:35eb2d60-2de0-4b00-9815-d5153edd0057:87653:1875449615415808:192.168.202.113@tcp:101:dd.0 00010000:00010000:2.0:1788572252.702474:0:87653:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035dc00/0xef30c340bfb4707b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326791 expref: -99 pid: 87653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572252.702488:0:87653:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035dc00/0xef30c340bfb4707b lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326791 expref: -99 pid: 87653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572252.702501:0:87653:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035dc00/0xef30c340bfb4707b lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8326791 expref: -99 pid: 87653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572252.702520:0:87653:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035dc00/0xef30c340bfb4707b lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326791 expref: -99 pid: 87653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572252.702535:0:87653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035dc00/0xef30c340bfb4707b lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8326791 expref: -99 pid: 87653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572252.702544:0:87653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035dc00/0xef30c340bfb4707b lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8326791 expref: -99 pid: 87653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572252.702552:0:87653:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035dc00/0xef30c340bfb4707b lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8326791 expref: -99 pid: 87653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1788572252.702566:0:87653:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada517fd500 x1875449615415808/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788572268 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1788572252.702582:0:87653:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035dc00/0xef30c340bfb4707b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326791 expref: -99 pid: 87653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572252.702597:0:87653:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:2.0:1788572252.702605:0:87653:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:2.0:1788572252.702641:0:87653:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:2.0:1788572252.702648:0:87653:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4707b 00000080:00010000:2.0:1788572252.702654:0:87653:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788572252.702658:0:87653:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4707b from it ffffab0ec16d3bf8 00010000:00010000:2.0:1788572252.702661:0:87653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035dc00/0xef30c340bfb4707b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8326791 expref: -99 pid: 87653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572252.702677:0:87653:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035dc00/0xef30c340bfb4707b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326791 expref: -99 pid: 87653 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788572252.702707:0:87653:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:2.0:1788572252.702736:0:87653:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f41.replay-single, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), file ffff9ada43c22180, open_flags 8000, mode 0 opened 0 00000080:00200000:2.0:1788572252.702744:0:87653:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f41.replay-single, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=open 00000002:00010000:2.0:1788572252.702773:0:87653:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f41.replay-single,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: open flags 0100001 00010000:00010000:2.0:1788572252.702830:0:87653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035c400/0xef30c340bfb47082 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 87653 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572252.702838:0:87653:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035c400/0xef30c340bfb47082 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 87653 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:2.0:1788572252.702850:0:87653:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1788572252.702881:0:87653:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada517fd500 x1875449615416064/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/131856 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1788572252.702891:0:87653:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035c400/0xef30c340bfb47082 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 87653 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572252.702915:0:87653:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517fd500 pname:cluuid:pid:xid:nid:opc:job dd:35eb2d60-2de0-4b00-9815-d5153edd0057:87653:1875449615416064:192.168.202.113@tcp:101:dd.0 00000100:00100000:2.0:1788572252.703164:0:87653:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada604786c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572252.714961:0:87653:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517fd500 pname:cluuid:pid:xid:nid:opc:job dd:35eb2d60-2de0-4b00-9815-d5153edd0057:87653:1875449615416064:192.168.202.113@tcp:101:dd.0 00010000:00010000:2.0:1788572252.714999:0:87653:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035c400/0xef30c340bfb47082 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b83267ad expref: -99 pid: 87653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572252.715035:0:87653:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035c400/0xef30c340bfb47082 lrc: 4/1,0 mode: --/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b83267ad expref: -99 pid: 87653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572252.715044:0:87653:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035c400/0xef30c340bfb47082 lrc: 4/1,0 mode: --/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b83267ad expref: -99 pid: 87653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572252.715058:0:87653:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035c400/0xef30c340bfb47082 lrc: 4/1,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b83267ad expref: -99 pid: 87653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572252.715080:0:87653:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035c400/0xef30c340bfb47082 lrc: 4/1,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b83267ad expref: -99 pid: 87653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572252.715097:0:87653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035c400/0xef30c340bfb47082 lrc: 5/2,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b83267ad expref: -99 pid: 87653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572252.715105:0:87653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035c400/0xef30c340bfb47082 lrc: 5/2,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b83267ad expref: -99 pid: 87653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572252.715113:0:87653:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035c400/0xef30c340bfb47082 lrc: 4/1,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b83267ad expref: -99 pid: 87653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1788572252.715126:0:87653:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x1 disposition=0x2b, status=0 req@ffff9ada517fd500 x1875449615416064/t219043332505(219043332505) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/664 e 0 to 0 dl 1788572268 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1788572252.715141:0:87653:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada517fd500 x1875449615416064/t219043332505(219043332505) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 664/664 e 0 to 0 dl 1788572268 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1788572252.715156:0:87653:0:(mdc_locks.c:939:mdc_finish_enqueue()) ### layout lock returned by: open, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035c400/0xef30c340bfb47082 lrc: 3/1,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b83267ad expref: -99 pid: 87653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:2.0:1788572252.715171:0:87653:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035c400/0xef30c340bfb47082 lrc: 3/1,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b83267ad expref: -99 pid: 87653 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572252.715183:0:87653:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8590033617/1603 (0 0) 00000080:00200000:2.0:1788572252.715191:0:87653:0:(namei.c:952:ll_intent_lock()) intent lock 1 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1788572252.715245:0:87653:0:(namei.c:147:ll_iget()) got inode: [0x2000182d1:0x643:0x0](ffff9ada58237448): rc = 0 00000080:00200000:2.0:1788572252.715261:0:87653:0:(file.c:6843:ll_layout_conf()) [0x2000182d1:0x643:0x0]: layout version change: 0 -> 0 00000080:00010000:2.0:1788572252.715266:0:87653:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x2000182d1:0x643:0x0](ffff9ada58237448) for lock 0xef30c340bfb47082 00000080:00010000:2.0:1788572252.715280:0:87653:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x2000182d1:0x643:0x0](ffff9ada58237448) 00000080:00200008:2.0:1788572252.715298:0:87653:0:(file.c:1090:ll_file_open()) START file f41.replay-single:[0x2000182d1:0x643:0x0](ffff9ada58237448), flags 100000 00000002:00100000:2.0:1788572252.715325:0:87653:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff9ada517fd500 x1875449615416064/t219043332505(219043332505) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788572268 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00000080:00010000:2.0:1788572252.715387:0:87653:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb47082 from it ffff9ada607811e0 00010000:00010000:2.0:1788572252.715392:0:87653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035c400/0xef30c340bfb47082 lrc: 3/1,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b83267ad expref: -99 pid: 87653 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572252.715402:0:87653:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035c400/0xef30c340bfb47082 lrc: 2/0,0 mode: PR/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b83267ad expref: -99 pid: 87653 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:2.0:1788572252.715433:0:87653:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x2000182d1:0x643:0x0](ffff9ada58237448), inode mode 81a4 mask 44 00000080:00200008:2.0:1788572252.715471:0:87653:0:(file.c:5214:ll_file_seek()) START file f41.replay-single:[0x2000182d1:0x643:0x0], offset: 0, type: SEEK_CUR 00000080:00200008:2.0:1788572252.715478:0:87653:0:(file.c:5245:ll_file_seek()) COMPLETED file f41.replay-single:[0x2000182d1:0x643:0x0], offset: 0, type: SEEK_CUR, rc = 0 00000080:00200008:2.0:1788572252.715604:0:87653:0:(file.c:2483:do_file_read_iter()) START file f41.replay-single:[0x2000182d1:0x643:0x0], ppos: 0, count: 4096 00000080:00200000:2.0:1788572252.715631:0:87653:0:(rw.c:1925:ll_readpage()) fast read pgno: 0 00000080:00200000:2.0:1788572252.715649:0:87653:0:(file.c:2090:ll_file_io_generic()) f41.replay-single: read ppos: 0, bytes: 4096 00000080:00200000:2.0:1788572252.715655:0:87653:0:(file.c:1848:ll_io_set_mirror()) f41.replay-single: desiginated mirror: 0 00000020:00200000:2.0:1788572252.715664:0:87653:0:(cl_io.c:214:cl_io_rw_init()) header@00000000cb95b7e8[0x110, 1, [0x2000182d1:0x643:0x0] hash] 00000020:00200000:2.0:1788572252.715666:0:87653:0:(cl_io.c:214:cl_io_rw_init()) io range: 1 [0, 4096) 0 0 00000080:00200000:2.0:1788572252.715671:0:87653:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x643:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00020000:00200000:2.0:1788572252.715692:0:87653:0:(lov_io.c:1277:lov_io_rw_iter_init()) stripe: 0 chunk: [0, 4096) 4096, 4096 00020000:00200000:2.0:1788572252.715697:0:87653:0:(lov_io.c:1085:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1788572252.715717:0:87653:0:(vvp_io.c:214:vvp_io_one_lock_index()) lock: 0 [0, 0] 00000008:00010000:2.0:1788572252.715735:0:87653:0:(lustre_osc.h:162:osc_enq2ldlm_flags()) flags: 0 00000008:00010000:2.0:1788572252.715742:0:87653:0:(osc_lock.c:1312:osc_lock_init()) ### lock ffff9ada607044b0, osc lock ffff9ada4880e720, flags 0x0 00010000:00010000:2.0:1788572252.715799:0:87653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000bf5c7bf0/0xef30c340bfb47089 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 87653 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572252.715809:0:87653:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x0 ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000bf5c7bf0/0xef30c340bfb47089 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 87653 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572252.715820:0:87653:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000bf5c7bf0/0xef30c340bfb47089 lrc: 3/1,0 mode: --/PR res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 87653 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572252.715843:0:87653:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517fc700 pname:cluuid:pid:xid:nid:opc:job dd:35eb2d60-2de0-4b00-9815-d5153edd0057:87653:1875449615416192:192.168.202.113@tcp:101:dd.0 00000100:00100000:3.0:1788572252.717205:0:87653:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada604786c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572252.731510:0:87653:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517fc700 pname:cluuid:pid:xid:nid:opc:job dd:35eb2d60-2de0-4b00-9815-d5153edd0057:87653:1875449615416192:192.168.202.113@tcp:101:dd.0 00010000:00010000:0.0:1788572252.731551:0:87653:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000bf5c7bf0/0xef30c340bfb47089 lrc: 4/1,0 mode: --/PR res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b83267bb expref: -99 pid: 87653 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572252.731575:0:87653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000bf5c7bf0/0xef30c340bfb47089 lrc: 4/1,0 mode: PR/PR res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b83267bb expref: -99 pid: 87653 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572252.731590:0:87653:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000bf5c7bf0/0xef30c340bfb47089 lrc: 4/1,0 mode: PR/PR res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b83267bb expref: -99 pid: 87653 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572252.731612:0:87653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000bf5c7bf0/0xef30c340bfb47089 lrc: 5/2,0 mode: PR/PR res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b83267bb expref: -99 pid: 87653 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00010000:0.0:1788572252.731626:0:87653:0:(osc_lock.c:187:osc_lock_lvb_update()) ### acquired size 4096, setting rss=4096; kms=4096, end=18446744073709551615 ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000bf5c7bf0/0xef30c340bfb47089 lrc: 4/2,0 mode: PR/PR res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b83267bb expref: -99 pid: 87653 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572252.731640:0:87653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000bf5c7bf0/0xef30c340bfb47089 lrc: 5/2,0 mode: PR/PR res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x810020000000000 nid: local remote: 0xdb7edb52b83267bb expref: -99 pid: 87653 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572252.731649:0:87653:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000bf5c7bf0/0xef30c340bfb47089 lrc: 4/1,0 mode: PR/PR res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x810020000000000 nid: local remote: 0xdb7edb52b83267bb expref: -99 pid: 87653 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000080:00200000:0.0:1788572252.731679:0:87653:0:(vvp_io.c:835:vvp_io_read_start()) f41.replay-single: read [0, 4096) 00000080:00200000:0.0:1788572252.731718:0:87653:0:(vvp_io.c:194:vvp_prep_size()) [0x2000182d1:0x643:0x0] updating i_size 4096 00000080:00200000:0.0:1788572252.731726:0:87653:0:(vvp_io.c:893:vvp_io_read_start()) count:4096 iocb pos:0 00000008:00010000:0.0:1788572252.731795:0:87653:0:(osc_cache.c:3059:osc_queue_sync_pages()) ### no dlm lock: extent: ffff9ada507f3240 00000008:00010000:0.0:1788572252.731846:0:83623:0:(osc_cache.c:2074:try_to_add_extent_for_io()) ### no dlm lock: extent: ffff9ada507f3240 00010000:00010000:0.0:1788572252.731946:0:83623:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 4095) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000bf5c7bf0/0xef30c340bfb47089 lrc: 4/1,0 mode: PR/PR res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000000000 nid: local remote: 0xdb7edb52b83267bb expref: -99 pid: 87653 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000008:00100000:0.0:1788572252.731993:0:83623:0:(osc_request.c:2000:osc_brw_prep_request()) brw rpc ffff9ada60a36a00 - object 0x280000401:2466 offset 0<>4096 00000100:00100000:0.0:1788572252.732058:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1788572252.733450:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a36a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615416320:192.168.202.113@tcp:3:dd.0 00000008:00010000:0.0:1788572252.741313:0:2384:0:(osc_cache.c:908:osc_extent_finish()) ### no dlm lock: extent: ffff9ada507f3240 00000020:00200000:0.0:1788572252.742442:0:2384:0:(cl_io.c:1849:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000008:00010000:0.0:1788572252.742462:0:2384:0:(osc_cache.c:497:__osc_extent_remove()) ### no dlm lock: extent: ffff9ada507f3240 00000100:00100000:0.0:1788572252.742505:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a36a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615416320:192.168.202.113@tcp:3:dd.0 00000020:00010000:3.0:1788572252.742963:0:87653:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff9ada5169a128 (ffff9ada60b0d008/0) at cl_lock_release():245 00000020:00010000:3.0:1788572252.742969:0:87653:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9ada5169a128 (ffff9ada60b0d008/0) at cl_lock_cancel():142 00000020:00010000:3.0:1788572252.742974:0:87653:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff9ada607044b0 (ffff9ada60b0d248/1) at cl_lock_cancel():142 00010000:00010000:3.0:1788572252.742986:0:87653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000bf5c7bf0/0xef30c340bfb47089 lrc: 4/1,0 mode: PR/PR res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x810020000000000 nid: local remote: 0xdb7edb52b83267bb expref: -99 pid: 87653 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572252.743052:0:87653:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000bf5c7bf0/0xef30c340bfb47089 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x800020000000000 nid: local remote: 0xdb7edb52b83267bb expref: -99 pid: 87653 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00000020:00010000:3.0:1788572252.743085:0:87653:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9ada5169a128 (ffff9ada60b0d008/0) at cl_lock_fini():72 00000020:00010000:3.0:1788572252.743088:0:87653:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff9ada607044b0 (ffff9ada60b0d008/1) at cl_lock_fini():72 00000080:00200000:3.0:1788572252.743120:0:87653:0:(vvp_io.c:533:vvp_io_advance()) advancing 4096 bytes 00000080:00200000:3.0:1788572252.743247:0:87653:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x643:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1788572252.743254:0:87653:0:(file.c:2238:ll_file_io_generic()) f41.replay-single: 1 io complete with rc: 0, result: 4096, restart: 0 00000080:00200000:3.0:1788572252.743260:0:87653:0:(file.c:2324:ll_file_io_generic()) iot: 1, result: 4096 00000080:00200008:3.0:1788572252.743302:0:87653:0:(file.c:479:ll_file_release()) START file f41.replay-single:[0x2000182d1:0x643:0x0](ffff9ada58237448), flags 100000 00000002:00100000:3.0:1788572252.743496:0:87653:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff9ada517fd500 x1875449615416064/t219043332505(219043332505) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788572268 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00000020:00100000:3.0:1788572252.743509:0:87653:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:3.0:1788572252.743515:0:87653:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada517a0000 x1875449615416576/t0(0) o35->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:23/10 lens 392/624 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:3.0:1788572252.743540:0:87653:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517a0000 pname:cluuid:pid:xid:nid:opc:job dd:35eb2d60-2de0-4b00-9815-d5153edd0057:87653:1875449615416576:192.168.202.113@tcp:35:dd.0 00000100:00100000:3.0:1788572252.743645:0:87653:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60419780 going to sleep for 11 seconds 00000100:00100000:0.0:1788572252.763212:0:87653:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517a0000 pname:cluuid:pid:xid:nid:opc:job dd:35eb2d60-2de0-4b00-9815-d5153edd0057:87653:1875449615416576:192.168.202.113@tcp:35:dd.0 00000002:00100000:0.0:1788572252.763252:0:87653:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff9ada517fd500 x1875449615416064/t219043332505(219043332505) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/664 e 0 to 0 dl 1788572268 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00010000:00010000:0.0:1788572253.048489:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572254.072126:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572255.095226:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572255.617741:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517a2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449615416704:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572255.617841:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517a2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615416960:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572255.617907:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517a1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615417216:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572255.617943:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1788572255.617950:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517a0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615416832:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572255.617977:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517a1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615417088:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572255.644999:0:2386:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 219043332506 gen 9 00000100:00100000:3.0:1788572255.645092:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517a0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615416832:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572255.645111:0:2386:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-ffff9ada58b0a000: committing for last_committed 8589935763 gen 5 00000100:00100000:3.0:1788572255.645125:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517a1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615417088:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572255.645136:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517a2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449615416704:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572255.645148:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517a2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615416960:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572255.645159:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517a1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615417216:192.168.202.113@tcp:400:kworker.0 00010000:00010000:1.0:1788572256.123768:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572257.143065:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572258.170770:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572259.191496:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572260.214951:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572260.732698:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a36a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449615417344:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572260.732793:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a36d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615417600:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572260.732824:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a37100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615417856:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572260.732882:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1788572260.732890:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a35180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615417472:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572260.732917:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60a36300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615417728:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572260.740546:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a36a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449615417344:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572260.740569:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a36d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615417600:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572260.740594:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a37100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615417856:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572260.740625:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a35180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615417472:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572260.740637:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60a36300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449615417728:192.168.202.113@tcp:400:kworker.0 00000001:02000400:2.0:1788572260.845802:0:88226:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == replay-single test 42: recovery after ost failure ===== 21:37:39 (1788572259) 00010000:00010000:3.0:1788572261.249753:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:3.0:1788572261.295359:0:88240:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1788572261.295374:0:88240:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:3.0:1788572261.295400:0:88240:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1788572261.295422:0:88240:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035dc00/0xef30c340bfb4707b lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326791 expref: -99 pid: 87653 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1788572261.295439:0:88240:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035dc00/0xef30c340bfb4707b lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326791 expref: -99 pid: 87653 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1788572261.295448:0:88240:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1788572261.295457:0:88240:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:3.0:1788572261.295461:0:88240:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4707b 00000080:00010000:3.0:1788572261.295469:0:88240:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4707b from it ffffab0ec0d17cb8 00010000:00010000:3.0:1788572261.295473:0:88240:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035dc00/0xef30c340bfb4707b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8326791 expref: -99 pid: 87653 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1788572261.295484:0:88240:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035dc00/0xef30c340bfb4707b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326791 expref: -99 pid: 87653 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1788572261.295515:0:88240:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:3.0:1788572261.295523:0:88240:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1788572261.295530:0:88240:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035dc00/0xef30c340bfb4707b lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326791 expref: -99 pid: 87653 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1788572261.295539:0:88240:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035dc00/0xef30c340bfb4707b lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326791 expref: -99 pid: 87653 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:3.0:1788572261.295546:0:88240:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1788572261.295550:0:88240:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:3.0:1788572261.295553:0:88240:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4707b 00000080:00010000:3.0:1788572261.295556:0:88240:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4707b from it ffffab0ec0d17bb0 00010000:00010000:3.0:1788572261.295560:0:88240:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035dc00/0xef30c340bfb4707b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8326791 expref: -99 pid: 87653 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:3.0:1788572261.295569:0:88240:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035dc00/0xef30c340bfb4707b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326791 expref: -99 pid: 87653 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:3.0:1788572261.295578:0:88240:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:3.0:1788572261.295585:0:88240:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100400 00000080:00200008:3.0:1788572261.295600:0:88240:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200008:3.0:1788572261.319524:0:88240:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1788572261.319536:0:88240:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:3.0:1788572261.319563:0:88240:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1788572261.319583:0:88240:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035dc00/0xef30c340bfb4707b lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326791 expref: -99 pid: 87653 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1788572261.319598:0:88240:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035dc00/0xef30c340bfb4707b lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326791 expref: -99 pid: 87653 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:3.0:1788572261.319606:0:88240:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1788572261.319614:0:88240:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:3.0:1788572261.319618:0:88240:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4707b 00000080:00010000:3.0:1788572261.319626:0:88240:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4707b from it ffffab0ec0d17cb8 00010000:00010000:3.0:1788572261.319630:0:88240:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035dc00/0xef30c340bfb4707b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8326791 expref: -99 pid: 87653 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1788572261.319640:0:88240:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035dc00/0xef30c340bfb4707b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326791 expref: -99 pid: 87653 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00200008:3.0:1788572261.319691:0:88240:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1788572261.319696:0:88240:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:3.0:1788572261.319704:0:88240:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1788572261.319711:0:88240:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035dc00/0xef30c340bfb4707b lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326791 expref: -99 pid: 87653 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:3.0:1788572261.319719:0:88240:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035dc00/0xef30c340bfb4707b lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326791 expref: -99 pid: 87653 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00200000:3.0:1788572261.319726:0:88240:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1788572261.319730:0:88240:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:3.0:1788572261.319732:0:88240:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4707b 00000080:00010000:3.0:1788572261.319735:0:88240:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4707b from it ffffab0ec0d17cb8 00010000:00010000:3.0:1788572261.319739:0:88240:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035dc00/0xef30c340bfb4707b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8326791 expref: -99 pid: 87653 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:3.0:1788572261.319746:0:88240:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035dc00/0xef30c340bfb4707b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8326791 expref: -99 pid: 87653 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:3.0:1788572261.319768:0:88240:0:(llite_lib.c:2903:ll_statfs()) VFS Op:sb=lustre (ffff9ada58b0a000) / proj=0 00000080:00000004:3.0:1788572261.319773:0:88240:0:(obd_class.h:1090:obd_statfs()) lustre-clilmv-ffff9ada58b0a000: age 2790, max_age 4177 00800000:00000004:3.0:1788572261.319785:0:88240:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: age 4145, max_age 4177 00000100:00100000:3.0:1788572261.319830:0:88240:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada517a1180 pname:cluuid:pid:xid:nid:opc:job df:35eb2d60-2de0-4b00-9815-d5153edd0057:88240:1875449615417984:192.168.202.113@tcp:41:df.0 00000100:00100000:3.0:1788572261.320104:0:88240:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60478300 going to sleep for 11 seconds 00000100:00100000:1.0:1788572261.327961:0:88240:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada517a1180 pname:cluuid:pid:xid:nid:opc:job df:35eb2d60-2de0-4b00-9815-d5153edd0057:88240:1875449615417984:192.168.202.113@tcp:41:df.0 00800000:00000004:1.0:1788572261.328010:0:88240:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: update ffff9ada4481d198 cache blocks 1802719/1916558 objects 523463/524040 00000080:00000004:1.0:1788572261.328018:0:88240:0:(obd_class.h:1116:obd_statfs()) lustre-clilmv-ffff9ada58b0a000: update ffff9ada44818458 cache blocks 1802719/1916558 objects 523463/524040 00000080:00000004:1.0:1788572261.328023:0:88240:0:(llite_lib.c:2755:ll_statfs_internal()) MDC blocks 1802719/1916558 objects 523463/524040 00010000:00010000:1.0:1788572262.273178:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572263.287072:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572264.311950:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788572270.970172:0:58077:0:(import.c:65:import_set_state_nolock()) ffff9ada48400800 lustre-OST0001_UUID: changing import state from FULL to CONNECTING 00000100:00080000:1.0:1788572270.970181:0:58077:0:(import.c:2147:ptlrpc_disconnect_and_idle_import()) lustre-OST0001-osc-ffff9ada58b0a000: disconnect after 21s idle 00000100:00080000:2.0:1788572271.014186:0:2382:0:(import.c:2057:ptlrpc_disconnect_idle_interpret()) @@@ inflight=1, refcount=3: rc = 0 req@ffff9ada45431f80 x1875449615586944/t0(0) o9->lustre-OST0001-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572280 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788572271.014204:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada48400800 lustre-OST0001_UUID: changing import state from CONNECTING to IDLE 00010000:00010000:3.0:1788572282.743877:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572283.774289:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:2.0:1788572284.657596:0:88443:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1788572284.657618:0:88443:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788572284.657657:0:88443:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788572284.657746:0:88443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8600/0xef30c340bfb49c49 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 88443 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572284.657759:0:88443:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8600/0xef30c340bfb49c49 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 88443 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572284.657773:0:88443:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8600/0xef30c340bfb49c49 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 88443 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572284.657820:0:88443:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60544000 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:88443:1875449615931776:192.168.202.113@tcp:101:lfs.0 00000100:00100000:2.0:1788572284.658081:0:88443:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada606cfe40 going to sleep for 11 seconds 00000100:00100000:1.0:1788572284.667947:0:88443:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 219043334106 gen 9 00000100:00100000:1.0:1788572284.668753:0:88443:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60544000 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:88443:1875449615931776:192.168.202.113@tcp:101:lfs.0 00010000:00010000:1.0:1788572284.668777:0:88443:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8600/0xef30c340bfb49c49 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b83300db expref: -99 pid: 88443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572284.668798:0:88443:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8600/0xef30c340bfb49c49 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b83300db expref: -99 pid: 88443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572284.668814:0:88443:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8600/0xef30c340bfb49c49 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b83300db expref: -99 pid: 88443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572284.668831:0:88443:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8600/0xef30c340bfb49c49 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b83300db expref: -99 pid: 88443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572284.668846:0:88443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8600/0xef30c340bfb49c49 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b83300db expref: -99 pid: 88443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572284.668881:0:88443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8600/0xef30c340bfb49c49 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b83300db expref: -99 pid: 88443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572284.668889:0:88443:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8600/0xef30c340bfb49c49 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b83300db expref: -99 pid: 88443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1788572284.668903:0:88443:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada60544000 x1875449615931776/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788572300 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1788572284.668918:0:88443:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8600/0xef30c340bfb49c49 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b83300db expref: -99 pid: 88443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572284.668932:0:88443:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1788572284.668940:0:88443:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1788572284.668973:0:88443:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:1.0:1788572284.668980:0:88443:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb49c49 00000080:00010000:1.0:1788572284.668986:0:88443:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:1.0:1788572284.668989:0:88443:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb49c49 from it ffffab0ec1c9fcb8 00010000:00010000:1.0:1788572284.668993:0:88443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8600/0xef30c340bfb49c49 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b83300db expref: -99 pid: 88443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572284.669002:0:88443:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8600/0xef30c340bfb49c49 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b83300db expref: -99 pid: 88443 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:1.0:1788572284.669846:0:88443:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 0, flags 0, foreign 0 00000080:00200000:1.0:1788572284.669940:0:88443:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:1.0:1788572284.669964:0:88443:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1788572284.669977:0:88443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8600/0xef30c340bfb49c49 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b83300db expref: -99 pid: 88443 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1788572284.669995:0:88443:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8600/0xef30c340bfb49c49 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b83300db expref: -99 pid: 88443 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572284.670004:0:88443:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1788572284.670009:0:88443:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:1.0:1788572284.670012:0:88443:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb49c49 00000080:00010000:1.0:1788572284.670016:0:88443:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb49c49 from it ffffab0ec1c9fcb8 00010000:00010000:1.0:1788572284.670018:0:88443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8600/0xef30c340bfb49c49 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b83300db expref: -99 pid: 88443 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1788572284.670024:0:88443:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8600/0xef30c340bfb49c49 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b83300db expref: -99 pid: 88443 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1788572284.670138:0:88443:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:1.0:1788572284.670191:0:88443:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1788572284.670197:0:88443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8600/0xef30c340bfb49c49 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b83300db expref: -99 pid: 88443 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1788572284.670203:0:88443:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8600/0xef30c340bfb49c49 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b83300db expref: -99 pid: 88443 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1788572284.670207:0:88443:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1788572284.670210:0:88443:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:1.0:1788572284.670211:0:88443:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb49c49 00000080:00010000:1.0:1788572284.670214:0:88443:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb49c49 from it ffffab0ec1c9fbb0 00010000:00010000:1.0:1788572284.670216:0:88443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8600/0xef30c340bfb49c49 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b83300db expref: -99 pid: 88443 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1788572284.670222:0:88443:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8600/0xef30c340bfb49c49 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b83300db expref: -99 pid: 88443 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00000080:00200000:1.0:1788572284.670228:0:88443:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:1.0:1788572284.670234:0:88443:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200080:1.0:1788572284.670314:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.670396:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00800000:00000004:1.0:1788572284.670407:0:88443:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: age 4179, max_age 4201 00000100:00100000:1.0:1788572284.670446:0:88443:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45535180 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:88443:1875449615931904:192.168.202.113@tcp:41:lfs.0 00000100:00100000:1.0:1788572284.670528:0:88443:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada4882ad80 going to sleep for 11 seconds 00000100:00100000:1.0:1788572284.676424:0:88443:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45535180 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:88443:1875449615931904:192.168.202.113@tcp:41:lfs.0 00800000:00000004:1.0:1788572284.676461:0:88443:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: update ffff9ada4481d198 cache blocks 317309/353529 objects 1022887/1024000 00000080:00200080:1.0:1788572284.676713:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.676733:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00800000:00000004:1.0:1788572284.676741:0:88443:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: age 4145, max_age 4201 00000100:00100000:1.0:1788572284.676796:0:88443:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45535180 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:88443:1875449615932032:192.168.202.113@tcp:41:lfs.0 00000100:00100000:1.0:1788572284.676934:0:88443:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada4882ad80 going to sleep for 11 seconds 00000100:00100000:1.0:1788572284.692661:0:88443:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45535180 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:88443:1875449615932032:192.168.202.113@tcp:41:lfs.0 00800000:00000004:1.0:1788572284.692701:0:88443:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: update ffff9ada44819e18 cache blocks 317392/353529 objects 1023736/1024000 00000080:00200080:1.0:1788572284.692892:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.692914:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.692924:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.692929:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.692935:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.692938:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.692942:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.692946:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.692950:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.692954:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.692958:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.692961:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.692965:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.692969:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.692973:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.692977:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.692982:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.692985:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.692989:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.692993:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.692997:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693000:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693004:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693007:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693011:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693015:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693019:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693023:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693027:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693032:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693036:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693039:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693043:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693047:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693051:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693055:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693059:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693062:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693066:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693069:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693073:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693076:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693080:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693084:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693087:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693091:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693095:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693098:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693102:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693106:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693110:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693113:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693117:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693121:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693126:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693129:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693133:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693137:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693141:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693158:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693165:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693168:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693174:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693178:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693182:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693186:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693190:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693193:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693197:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693201:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693205:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693209:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693213:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693216:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693220:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693223:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693227:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693231:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693235:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693239:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693243:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693247:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693250:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693254:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693259:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693262:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693266:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693269:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693273:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693277:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693281:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693285:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693288:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693291:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693295:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693299:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693303:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693306:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693310:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693314:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693318:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693321:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693326:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693329:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693345:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693350:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693355:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693359:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693363:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693367:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693372:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693376:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693380:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693383:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693387:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693391:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693395:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693399:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693414:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693419:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693424:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693428:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693432:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693435:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693439:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00800000:00000080:1.0:1788572284.693443:0:88443:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.693448:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.693454:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00020000:00000004:1.0:1788572284.693464:0:88443:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: age 4145, max_age 4201 00000100:00100000:1.0:1788572284.693522:0:88443:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45535180 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:88443:1875449615932160:192.168.202.113@tcp:13:lfs.0 00000100:00100000:1.0:1788572284.694033:0:88443:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada4882ad80 going to sleep for 11 seconds 00000100:00100000:1.0:1788572284.708251:0:88443:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45535180 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:88443:1875449615932160:192.168.202.113@tcp:13:lfs.0 00020000:00000004:1.0:1788572284.708296:0:88443:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: update ffff9ada4354d198 cache blocks 901356/958279 objects 261313/262144 00000080:00200080:1.0:1788572284.709020:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.709043:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00020000:00000004:1.0:1788572284.709054:0:88443:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: age 4145, max_age 4201 00000100:00080000:1.0:1788572284.709068:0:88443:0:(import.c:545:import_select_connection()) lustre-OST0001-osc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 1046 00000100:00080000:1.0:1788572284.709082:0:88443:0:(import.c:665:import_select_connection()) lustre-OST0001-osc-ffff9ada58b0a000: import ffff9ada48400800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572284.709095:0:88443:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683394 ocd_grant: 491585536, lost: 0. 00000100:00100000:1.0:1788572284.709126:0:88443:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada45535880 x1875449615932288/t0(0) o8->lustre-OST0001-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572284.709180:0:88443:0:(client.c:1846:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != CONNECTING) req@ffff9ada45535180 x1875449615932416/t0(0) o13->lustre-OST0001-osc-ffff9ada58b0a000@192.168.202.113@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572284.709190:0:88443:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada4882ad80 going to sleep for 0 seconds 00000100:00100000:2.0:1788572284.709317:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45535880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615932288:192.168.202.113@tcp:8:lfs.0 00000100:00100000:0.0:1788572284.712251:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-ffff9ada58b0a000: committing for last_committed 4294967635 gen 6 00000100:00080000:0.0:1788572284.712273:0:2382:0:(import.c:1176:ptlrpc_connect_interpret()) lustre-OST0001-osc-ffff9ada58b0a000: connect to target with instance 1 00000100:00080000:0.0:1788572284.712285:0:2382:0:(import.c:1000:ptlrpc_connect_set_flags()) lustre-OST0001-osc-ffff9ada58b0a000: Resetting ns_connect_flags to server flags: 0xa0427af2e3440038 00000100:00080000:0.0:1788572284.712300:0:2382:0:(import.c:1257:ptlrpc_connect_interpret()) connected to replayable target: lustre-OST0001_UUID 00000100:00080000:0.0:1788572284.712304:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada48400800 lustre-OST0001_UUID: changing import state from CONNECTING to FULL 00000080:00000004:0.0:1788572284.712316:0:2382:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0xa0427af2e3440038 -> 0xa0427af2e3440038 00000080:00080000:0.0:1788572284.712323:0:2382:0:(lcommon_misc.c:56:cl_init_ea_size()) updating def/max_easize: 72/65536 00000100:00080000:0.0:1788572284.712353:0:2382:0:(recover.c:191:ptlrpc_wake_delayed()) @@@ waking (set 00000000a6e1762d): req@ffff9ada45535180 x1875449615932416/t0(0) o13->lustre-OST0001-osc-ffff9ada58b0a000@192.168.202.113@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572284.712989:0:88443:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada4882ad80 going to sleep for 11 seconds 00000100:00100000:0.0:1788572284.715909:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45535880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449615932288:192.168.202.113@tcp:8:lfs.0 00000100:00100000:1.0:1788572284.720207:0:88443:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45535180 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:88443:1875449615932416:192.168.202.113@tcp:13:lfs.0 00020000:00000004:1.0:1788572284.720247:0:88443:0:(obd_class.h:1116:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: update ffff9ada43549e18 cache blocks 901363/958279 objects 261350/262144 00000080:00200080:1.0:1788572284.720710:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.720734:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.720743:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.720748:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.720752:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.720756:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.720761:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.720765:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.720770:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.720774:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.720778:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.720782:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.720797:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.720801:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.720806:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.720810:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.720815:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.720819:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.721432:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.721445:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.721452:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.721456:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.721461:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.721464:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.721469:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.721473:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.721477:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.721480:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.721485:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.721488:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.721492:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.721496:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.721499:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.721502:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.721738:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.721745:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.721749:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.721752:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.721756:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.721759:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.721763:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.721766:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.721770:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.721774:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.721778:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.721782:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.721797:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.721802:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.721806:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.721810:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.721813:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.721817:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.721821:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.722109:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.722118:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.722122:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.722127:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.722130:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.722135:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.722138:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.722142:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.722146:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.722150:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.722153:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.722157:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.722161:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.722165:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.722169:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.722173:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.722176:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.722180:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.722184:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.722189:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.722192:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.722196:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.722200:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.722204:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.722207:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.722211:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.722215:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.722219:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.722223:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.722227:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.722231:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.722235:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.722239:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.722243:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.722246:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.722251:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.722254:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.722257:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.722260:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.722264:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.722268:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.722272:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.722275:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.722279:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.722283:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.722287:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.722291:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.722295:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.722298:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.722303:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.722306:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.722310:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.722313:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.722318:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.722321:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.722325:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.722329:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.723452:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.723476:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.723489:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.723494:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.723498:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.723501:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.723515:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.723519:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.723523:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.723527:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.723531:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.723534:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.723538:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.723551:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200080:1.0:1788572284.723556:0:88443:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe0bff4200 00020000:00000080:1.0:1788572284.723561:0:88443:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada454a4800 uarg=0000000000000000 00000080:00200008:1.0:1788572284.723578:0:88443:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00010000:00010000:2.0:1788572284.791470:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:2.0:1788572284.814499:0:88450:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788572284.814766:0:88450:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788572284.814802:0:88450:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8600/0xef30c340bfb49c49 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b83300db expref: -99 pid: 88443 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00010000:00010000:2.0:1788572284.816949:0:88450:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8600/0xef30c340bfb49c49 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b83300db expref: -99 pid: 88443 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00000080:00200000:2.0:1788572284.816975:0:88450:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788572284.816989:0:88450:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788572284.816993:0:88450:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb49c49 00000080:00010000:2.0:1788572284.817005:0:88450:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb49c49 from it ffffab0ec1ce7bd0 00010000:00010000:2.0:1788572284.817010:0:88450:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8600/0xef30c340bfb49c49 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b83300db expref: -99 pid: 88443 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00010000:00010000:2.0:1788572284.817022:0:88450:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8600/0xef30c340bfb49c49 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b83300db expref: -99 pid: 88443 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00200000:2.0:1788572284.817318:0:88450:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:2.0:1788572284.817474:0:88450:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=fsa-oleg213-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), flags=1536 00000080:00200000:2.0:1788572284.817481:0:88450:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788572284.817501:0:88450:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788572284.817529:0:88450:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8600/0xef30c340bfb49c49 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b83300db expref: -99 pid: 88443 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:2.0:1788572284.818002:0:88450:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8600/0xef30c340bfb49c49 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b83300db expref: -99 pid: 88443 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00200000:2.0:1788572284.818012:0:88450:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788572284.818018:0:88450:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788572284.818021:0:88450:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb49c49 00000080:00010000:2.0:1788572284.818026:0:88450:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb49c49 from it ffffab0ec1ce7cd0 00010000:00010000:2.0:1788572284.818029:0:88450:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8600/0xef30c340bfb49c49 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b83300db expref: -99 pid: 88443 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:2.0:1788572284.818099:0:88450:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8600/0xef30c340bfb49c49 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b83300db expref: -99 pid: 88443 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:2.0:1788572284.818110:0:88450:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:2.0:1788572284.818120:0:88450:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788572284.818131:0:88450:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788572284.818139:0:88450:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8600/0xef30c340bfb49c49 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b83300db expref: -99 pid: 88443 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:2.0:1788572284.818146:0:88450:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8600/0xef30c340bfb49c49 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b83300db expref: -99 pid: 88443 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:2.0:1788572284.818157:0:88450:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788572284.818161:0:88450:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788572284.818163:0:88450:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb49c49 00000080:00010000:2.0:1788572284.818166:0:88450:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb49c49 from it ffffab0ec1ce7db0 00010000:00010000:2.0:1788572284.818169:0:88450:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8600/0xef30c340bfb49c49 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b83300db expref: -99 pid: 88443 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:2.0:1788572284.818175:0:88450:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8600/0xef30c340bfb49c49 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b83300db expref: -99 pid: 88443 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000080:00200000:2.0:1788572284.818244:0:88450:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:2.0:1788572284.818250:0:88450:0:(namei.c:2062:ll_create_nd()) VFS Op:name=fsa-oleg213-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), flags=33188, excl=1 00000080:00200000:2.0:1788572284.818255:0:88450:0:(namei.c:2017:ll_mknod()) VFS Op:name=fsa-oleg213-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9ada58125b88) mode 100644 dev 0 00010000:00010000:2.0:1788572284.818286:0:88450:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8600/0xef30c340bfb49c49 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b83300db expref: -99 pid: 88443 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:2.0:1788572284.820676:0:88450:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8600/0xef30c340bfb49c49 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b83300db expref: -99 pid: 88443 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:2.0:1788572284.820761:0:88450:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1788572284.820766:0:88450:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8600/0xef30c340bfb49c49 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b83300db expref: -99 pid: 88443 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000020:00100000:2.0:1788572284.820812:0:88450:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1788572284.821109:0:88450:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada60545c00 x1875449615932672/t0(0) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788572284.821149:0:88450:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60545c00 pname:cluuid:pid:xid:nid:opc:job mcreate:35eb2d60-2de0-4b00-9815-d5153edd0057:88450:1875449615932672:192.168.202.113@tcp:36:mcreate.0 00000100:00100000:2.0:1788572284.821325:0:88450:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada606cf300 going to sleep for 11 seconds 00000100:00100000:0.0:1788572284.849512:0:88450:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60545c00 pname:cluuid:pid:xid:nid:opc:job mcreate:35eb2d60-2de0-4b00-9815-d5153edd0057:88450:1875449615932672:192.168.202.113@tcp:36:mcreate.0 00000080:00200000:0.0:1788572284.849643:0:88450:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x2000182d1:0x965:0x0](ffff9ada4629e3c8) 00000080:00200000:0.0:1788572284.849701:0:88450:0:(namei.c:2070:ll_create_nd()) VFS Op:name=fsa-oleg213-client.virtnet, unhashed 1 00000080:00200000:3.0:1788572284.862621:0:88453:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:3.0:1788572284.862664:0:88453:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1788572284.862802:0:88453:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5e00/0xef30c340bfb49c50 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 88453 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572284.862811:0:88453:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5e00/0xef30c340bfb49c50 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 88453 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572284.863132:0:88453:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5e00/0xef30c340bfb49c50 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 88453 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572284.863160:0:88453:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455a6680 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:88453:1875449615932800:192.168.202.113@tcp:101:rm.0 00000100:00100000:3.0:1788572284.863279:0:88453:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada4352f900 going to sleep for 11 seconds 00000100:00100000:3.0:1788572284.867735:0:88453:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455a6680 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:88453:1875449615932800:192.168.202.113@tcp:101:rm.0 00010000:00010000:3.0:1788572284.867774:0:88453:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5e00/0xef30c340bfb49c50 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b83300fe expref: -99 pid: 88453 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572284.867820:0:88453:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5e00/0xef30c340bfb49c50 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b83300fe expref: -99 pid: 88453 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572284.867835:0:88453:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5e00/0xef30c340bfb49c50 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b83300fe expref: -99 pid: 88453 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572284.867848:0:88453:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5e00/0xef30c340bfb49c50 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b83300fe expref: -99 pid: 88453 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572284.867996:0:88453:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5e00/0xef30c340bfb49c50 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b83300fe expref: -99 pid: 88453 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572284.868127:0:88453:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5e00/0xef30c340bfb49c50 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b83300fe expref: -99 pid: 88453 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572284.868138:0:88453:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5e00/0xef30c340bfb49c50 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b83300fe expref: -99 pid: 88453 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1788572284.868153:0:88453:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada455a6680 x1875449615932800/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788572300 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1788572284.868171:0:88453:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5e00/0xef30c340bfb49c50 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b83300fe expref: -99 pid: 88453 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572284.868187:0:88453:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:3.0:1788572284.868195:0:88453:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1788572284.868230:0:88453:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:3.0:1788572284.868238:0:88453:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb49c50 00000080:00010000:3.0:1788572284.868245:0:88453:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:3.0:1788572284.868249:0:88453:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb49c50 from it ffffab0ec1c3fb58 00010000:00010000:3.0:1788572284.868252:0:88453:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5e00/0xef30c340bfb49c50 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b83300fe expref: -99 pid: 88453 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572284.868262:0:88453:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5e00/0xef30c340bfb49c50 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b83300fe expref: -99 pid: 88453 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1788572284.868292:0:88453:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:3.0:1788572284.868317:0:88453:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=fsa-oleg213-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), flags=8 00000080:00200000:3.0:1788572284.868322:0:88453:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=fsa-oleg213-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=getattr 00000002:00010000:3.0:1788572284.868353:0:88453:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: fsa-oleg213-client.virtnet,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 01 00010000:00010000:3.0:1788572284.868404:0:88453:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d4800/0xef30c340bfb49c57 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 88453 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572284.868412:0:88453:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d4800/0xef30c340bfb49c57 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 88453 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572284.868422:0:88453:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d4800/0xef30c340bfb49c57 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 88453 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572284.868447:0:88453:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455a6680 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:88453:1875449615932928:192.168.202.113@tcp:101:rm.0 00000100:00100000:3.0:1788572284.868694:0:88453:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada4352f900 going to sleep for 11 seconds 00000100:00100000:0.0:1788572284.886625:0:88453:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455a6680 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:88453:1875449615932928:192.168.202.113@tcp:101:rm.0 00010000:00010000:0.0:1788572284.886665:0:88453:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d4800/0xef30c340bfb49c57 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833011a expref: -99 pid: 88453 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572284.886909:0:88453:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d4800/0xef30c340bfb49c57 lrc: 4/1,0 mode: --/PR res: [0x2000182d1:0x965:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833011a expref: -99 pid: 88453 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572284.886922:0:88453:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d4800/0xef30c340bfb49c57 lrc: 4/1,0 mode: --/PR res: [0x2000182d1:0x965:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833011a expref: -99 pid: 88453 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572284.886938:0:88453:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d4800/0xef30c340bfb49c57 lrc: 4/1,0 mode: PR/PR res: [0x2000182d1:0x965:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833011a expref: -99 pid: 88453 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572284.886953:0:88453:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d4800/0xef30c340bfb49c57 lrc: 4/1,0 mode: PR/PR res: [0x2000182d1:0x965:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833011a expref: -99 pid: 88453 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572284.886977:0:88453:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d4800/0xef30c340bfb49c57 lrc: 5/2,0 mode: PR/PR res: [0x2000182d1:0x965:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833011a expref: -99 pid: 88453 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572284.886987:0:88453:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d4800/0xef30c340bfb49c57 lrc: 5/2,0 mode: PR/PR res: [0x2000182d1:0x965:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833011a expref: -99 pid: 88453 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572284.886995:0:88453:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d4800/0xef30c340bfb49c57 lrc: 4/1,0 mode: PR/PR res: [0x2000182d1:0x965:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833011a expref: -99 pid: 88453 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788572284.887078:0:88453:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada455a6680 x1875449615932928/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 600/584 e 0 to 0 dl 1788572300 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788572284.887095:0:88453:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d4800/0xef30c340bfb49c57 lrc: 3/1,0 mode: PR/PR res: [0x2000182d1:0x965:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833011a expref: -99 pid: 88453 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572284.887110:0:88453:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8590033617/2405 (0 0) 00000080:00200000:0.0:1788572284.887118:0:88453:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1788572284.887144:0:88453:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x2000182d1:0x965:0x0], updating i_size 0 00000080:00200000:0.0:1788572284.887148:0:88453:0:(namei.c:147:ll_iget()) got inode: [0x2000182d1:0x965:0x0](ffff9ada4629e3c8): rc = 0 00000080:00200000:0.0:1788572284.887168:0:88453:0:(file.c:6843:ll_layout_conf()) [0x2000182d1:0x965:0x0]: layout version change: 4294967294 -> 4294967295 00000080:00010000:0.0:1788572284.887176:0:88453:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x2000182d1:0x965:0x0](ffff9ada4629e3c8) for lock 0xef30c340bfb49c57 00000080:00010000:0.0:1788572284.887189:0:88453:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x2000182d1:0x965:0x0](ffff9ada4629e3c8) 00000080:00010000:0.0:1788572284.887194:0:88453:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb49c57 from it ffffab0ec1c3fb20 00010000:00010000:0.0:1788572284.887199:0:88453:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d4800/0xef30c340bfb49c57 lrc: 3/1,0 mode: PR/PR res: [0x2000182d1:0x965:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b833011a expref: -99 pid: 88453 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572284.887209:0:88453:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d4800/0xef30c340bfb49c57 lrc: 2/0,0 mode: PR/PR res: [0x2000182d1:0x965:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b833011a expref: -99 pid: 88453 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:0.0:1788572284.887259:0:88453:0:(file.c:6296:ll_getattr_dentry()) START file fsa-oleg213-client.virtnet:[0x2000182d1:0x965:0x0](ffff9ada4629e3c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1788572284.887268:0:88453:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x2000182d1:0x965:0x0](ffff9ada4629e3c8),name=fsa-oleg213-client.virtnet 00000002:00010000:0.0:1788572284.887295:0:88453:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x2000182d1:0x965:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572284.887306:0:88453:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d4800/0xef30c340bfb49c57 lrc: 2/1,0 mode: PR/PR res: [0x2000182d1:0x965:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b833011a expref: -99 pid: 88453 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788572284.887319:0:88453:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d4800/0xef30c340bfb49c57 lrc: 2/1,0 mode: PR/PR res: [0x2000182d1:0x965:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b833011a expref: -99 pid: 88453 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572284.887327:0:88453:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572284.887333:0:88453:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x2000182d1:0x965:0x0](ffff9ada4629e3c8) 00000080:00010000:0.0:1788572284.887336:0:88453:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x2000182d1:0x965:0x0](ffff9ada4629e3c8) for lock 0xef30c340bfb49c57 00000080:00010000:0.0:1788572284.887340:0:88453:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb49c57 from it ffffab0ec1c3fcb8 00010000:00010000:0.0:1788572284.887343:0:88453:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d4800/0xef30c340bfb49c57 lrc: 3/1,0 mode: PR/PR res: [0x2000182d1:0x965:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b833011a expref: -99 pid: 88453 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788572284.887352:0:88453:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d4800/0xef30c340bfb49c57 lrc: 2/0,0 mode: PR/PR res: [0x2000182d1:0x965:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b833011a expref: -99 pid: 88453 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788572284.887492:0:88453:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572284.887502:0:88453:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572284.887638:0:88453:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5e00/0xef30c340bfb49c50 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b83300fe expref: -99 pid: 88453 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788572284.887649:0:88453:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5e00/0xef30c340bfb49c50 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b83300fe expref: -99 pid: 88453 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572284.887664:0:88453:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572284.887669:0:88453:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572284.887672:0:88453:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb49c50 00000080:00010000:0.0:1788572284.887677:0:88453:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb49c50 from it ffffab0ec1c3fc10 00010000:00010000:0.0:1788572284.887680:0:88453:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5e00/0xef30c340bfb49c50 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b83300fe expref: -99 pid: 88453 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788572284.888156:0:88453:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5e00/0xef30c340bfb49c50 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b83300fe expref: -99 pid: 88453 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788572284.888169:0:88453:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:0.0:1788572284.888183:0:88453:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=fsa-oleg213-client.virtnet, flags=0 00000080:00200000:0.0:1788572284.888191:0:88453:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572284.888201:0:88453:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572284.888209:0:88453:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5e00/0xef30c340bfb49c50 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b83300fe expref: -99 pid: 88453 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788572284.888219:0:88453:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5e00/0xef30c340bfb49c50 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b83300fe expref: -99 pid: 88453 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788572284.888227:0:88453:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572284.888232:0:88453:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572284.888234:0:88453:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb49c50 00000080:00010000:0.0:1788572284.888239:0:88453:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb49c50 from it ffffab0ec1c3fd68 00010000:00010000:0.0:1788572284.888242:0:88453:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5e00/0xef30c340bfb49c50 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b83300fe expref: -99 pid: 88453 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788572284.888255:0:88453:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5e00/0xef30c340bfb49c50 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b83300fe expref: -99 pid: 88453 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1788572284.888264:0:88453:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:0.0:1788572284.888269:0:88453:0:(namei.c:2351:ll_unlink()) VFS Op:name=fsa-oleg213-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9ada58125b88) 00010000:00010000:0.0:1788572284.888297:0:88453:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5e00/0xef30c340bfb49c50 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b83300fe expref: -99 pid: 88453 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1788572284.888948:0:88453:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d4800/0xef30c340bfb49c57 lrc: 2/0,0 mode: PR/PR res: [0x2000182d1:0x965:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0xdb7edb52b833011a expref: -99 pid: 88453 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788572284.888966:0:88453:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x965:0x0]: losing layout lock 00010000:00010000:0.0:1788572284.889086:0:88453:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d4800/0xef30c340bfb49c57 lrc: 1/0,0 mode: --/PR res: [0x2000182d1:0x965:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0xdb7edb52b833011a expref: -99 pid: 88453 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788572284.889097:0:88453:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5e00/0xef30c340bfb49c50 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b83300fe expref: -99 pid: 88453 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1788572284.889104:0:88453:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:0.0:1788572284.889108:0:88453:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d4800/0xef30c340bfb49c57 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x965:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0xdb7edb52b833011a expref: -99 pid: 88453 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788572284.889132:0:88453:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5e00/0xef30c340bfb49c50 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b83300fe expref: -99 pid: 88453 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:0.0:1788572284.889157:0:88453:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1788572284.889172:0:88453:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada51788a80 x1875449615933184/t0(0) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 512/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788572284.889203:0:88453:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada51788a80 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:88453:1875449615933184:192.168.202.113@tcp:36:rm.0 00000100:00100000:0.0:1788572284.889408:0:88453:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6076ce40 going to sleep for 11 seconds 00000100:00100000:0.0:1788572284.927030:0:88453:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada51788a80 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:88453:1875449615933184:192.168.202.113@tcp:36:rm.0 00000080:00200000:0.0:1788572284.927121:0:88453:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x965:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:0.0:1788572284.927135:0:88453:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x965:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1788572284.927145:0:88453:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x965:0x0](ffff9ada4629e3c8) 00010000:00010000:3.0:1788572285.848261:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572286.327207:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60546300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449615933312:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572286.327216:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60544000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449615933440:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572286.329716:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60544a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449615933696:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572286.333957:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1788572286.334053:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60544e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449615933824:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572286.345194:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60547800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615933568:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572286.362478:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60544a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449615933696:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572286.366481:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60546300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449615933312:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572286.366510:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60547800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615933568:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572286.366686:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60544e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449615933824:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572286.366703:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60544000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449615933440:192.168.202.113@tcp:400:kworker.0 00010000:00010000:0.0:1788572286.840224:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572287.863527:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572288.887110:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572289.911220:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572290.938176:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788572291.447173:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60546300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615934080:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572291.447291:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60546680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615934336:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572291.447325:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:2.0:1788572291.447332:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60547800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449615933952:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572291.447374:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60545f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615934208:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572291.447395:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60544380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615934464:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572291.472943:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60545f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615934208:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572291.472964:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60544380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615934464:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572291.472976:0:2386:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 219043334108 gen 9 00000100:00100000:3.0:1788572291.473001:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60546300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615934080:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572291.475763:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60547800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449615933952:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572291.479722:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60546680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615934336:192.168.202.113@tcp:400:kworker.0 00010000:00010000:3.0:1788572291.959493:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572292.984498:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572294.007654:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:0.0:1788572294.413175:0:88520:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: local REPLAY BARRIER on lustre-OST0000 00010000:00010000:3.0:1788572295.033479:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572296.055784:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788572296.579582:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60547800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449615934592:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572296.579676:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60545880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615934848:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572296.579705:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60546a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615935104:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572296.579737:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1788572296.579743:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60545500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615934720:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572296.579777:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60547b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615934976:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572296.596587:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60545880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615934848:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572296.596756:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60545500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615934720:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572296.596781:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60547b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615934976:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572296.596791:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60547800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449615934592:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572296.596802:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60546a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449615935104:192.168.202.113@tcp:400:kworker.0 00010000:00010000:0.0:1788572302.200059:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572303.224035:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572304.247471:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572305.271016:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572305.271026:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572305.271032:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572305.271033:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572305.271043:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572305.271045:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572305.271049:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572305.271050:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572305.271152:0:15205:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572305.271157:0:15205:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572305.271160:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572306.295059:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572306.295068:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572306.295073:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572306.295077:0:365:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:2.0:1788572306.295080:0:365:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:2.0:1788572306.295084:0:365:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572306.295088:0:365:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:2.0:1788572306.295091:0:365:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572306.295094:0:365:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572306.295096:0:365:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572306.295100:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572306.295101:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572306.295105:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572306.295106:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572306.295902:0:15204:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572306.295907:0:15204:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572306.295912:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1788572306.809514:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788572306.809527:0:77466:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788572306.809541:0:77466:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada60626680. 00000020:00000040:2.0:1788572306.809549:0:77466:0:(genops.c:1210:class_import_get()) import ffff9ada58a62800 refcount=4 obd=MGC192.168.202.113@tcp 00000100:00000001:2.0:1788572306.809555:0:77466:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788572306.809558:0:77466:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788572306.809565:0:77466:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788572306.809575:0:77466:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada428be000. 00000100:00000001:2.0:1788572306.809584:0:77466:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572306.809599:0:77466:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS req@ffff9ada60626680 x1875449616089344/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788572306.809611:0:77466:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788572306.809616:0:77466:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572306.809623:0:77466:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d938d824] to pc [ptlrpcd_01_00+0] req@ffff9ada60626680 x1875449616089344/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.809655:0:77466:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572306.809658:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788572306.809662:0:77466:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788572306.809665:0:77466:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada60624380. 00000020:00000040:2.0:1788572306.809668:0:77466:0:(genops.c:1210:class_import_get()) import ffff9ada51140000 refcount=404 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000100:00000001:2.0:1788572306.809671:0:77466:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788572306.809672:0:77466:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788572306.809674:0:77466:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788572306.809677:0:77466:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada428be100. 00000100:00000001:2.0:1788572306.809681:0:77466:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572306.809686:0:77466:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID req@ffff9ada60624380 x1875449616089472/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788572306.809693:0:77466:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788572306.809695:0:77466:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572306.809699:0:77466:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000014a47584] to pc [ptlrpcd_01_01+1] req@ffff9ada60624380 x1875449616089472/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.809707:0:77466:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572306.809710:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788572306.809713:0:77466:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788572306.809717:0:77466:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada60626a00. 00000020:00000040:2.0:1788572306.809719:0:77466:0:(genops.c:1210:class_import_get()) import ffff9ada58b08000 refcount=4 obd=lustre-MDT0001-mdc-ffff9ada58b0a000 00000100:00000001:2.0:1788572306.809721:0:77466:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788572306.809723:0:77466:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788572306.809724:0:77466:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788572306.809728:0:77466:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada428be400. 00000100:00000001:2.0:1788572306.809731:0:77466:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572306.809735:0:77466:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID req@ffff9ada60626a00 x1875449616089600/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788572306.809743:0:77466:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788572306.809745:0:77466:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572306.809748:0:77466:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000006ee75d69] to pc [ptlrpcd_01_00+0] req@ffff9ada60626a00 x1875449616089600/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.809755:0:77466:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572306.809757:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788572306.809760:0:77466:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788572306.809764:0:77466:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada60627100. 00000020:00000040:2.0:1788572306.809766:0:77466:0:(genops.c:1210:class_import_get()) import ffff9ada48407800 refcount=3 obd=lustre-OST0000-osc-ffff9ada58b0a000 00000100:00000001:2.0:1788572306.809768:0:77466:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788572306.809769:0:77466:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788572306.809771:0:77466:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788572306.809775:0:77466:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada428be300. 00000100:00000001:2.0:1788572306.809777:0:77466:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572306.809781:0:77466:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0000_UUID req@ffff9ada60627100 x1875449616089728/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788572306.809788:0:77466:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788572306.809790:0:77466:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572306.809793:0:77466:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000742328b2] to pc [ptlrpcd_01_01+1] req@ffff9ada60627100 x1875449616089728/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.809953:0:77466:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572306.809955:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788572306.809959:0:77466:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:2.0:1788572306.809962:0:77466:0:(import.c:2112:ptlrpc_disconnect_and_idle_import()) Process entered 00000100:00000001:2.0:1788572306.809966:0:77466:0:(import.c:1806:ptlrpc_disconnect_prep_req()) Process entered 00000100:00000010:2.0:1788572306.809969:0:77466:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada60624700. 00000020:00000040:2.0:1788572306.809972:0:77466:0:(genops.c:1210:class_import_get()) import ffff9ada48400800 refcount=3 obd=lustre-OST0001-osc-ffff9ada58b0a000 00000100:00000001:2.0:1788572306.809974:0:77466:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788572306.809976:0:77466:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788572306.809978:0:77466:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788572306.809984:0:77466:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada5379ef00. 00000100:00000001:2.0:1788572306.809988:0:77466:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572306.809990:0:77466:0:(import.c:1844:ptlrpc_disconnect_prep_req()) Process leaving (rc=18446632861443442432 : -111212266109184 : ffff9ada60624700) 00000100:00080000:2.0:1788572306.809994:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada48400800 lustre-OST0001_UUID: changing import state from FULL to CONNECTING 00000100:00080000:2.0:1788572306.809998:0:77466:0:(import.c:2147:ptlrpc_disconnect_and_idle_import()) lustre-OST0001-osc-ffff9ada58b0a000: disconnect after 22s idle 00000100:00000001:2.0:1788572306.810000:0:77466:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788572306.810002:0:77466:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572306.810007:0:77466:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ff5b9a0d] to pc [ptlrpcd_rcv+-1] req@ffff9ada60624700 x1875449616089856/t0(0) o9->lustre-OST0001-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572306.810072:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788572306.810076:0:2382:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788572306.810079:0:2382:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788572306.810086:0:2382:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada60624700 x1875449616089856/t0(0) o9->lustre-OST0001-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572306.810094:0:2382:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788572306.810096:0:2382:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572306.810101:0:2382:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788572306.810104:0:2382:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788572306.810106:0:2382:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788572306.810110:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616089856:192.168.202.113@tcp:9:kworker.0 00000100:00000001:0.0:1788572306.810115:0:2382:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788572306.810122:0:2382:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788572306.810126:0:2382:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572306.810129:0:2382:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788572306.810135:0:2382:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada456bba00. 02000000:00000001:0.0:1788572306.810137:0:2382:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788572306.810147:0:2382:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:0.0:1788572306.810152:0:2382:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada60acaf20. 00000400:00000010:0.0:1788572306.810161:0:2382:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada5176a088. 00000100:00000200:0.0:1788572306.810169:0:2382:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616089856, portal 4 00000100:00000001:0.0:1788572306.810172:0:2382:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788572306.810174:0:2382:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632861443442432 : -111212266109184 : ffff9ada60624700) 00000100:00000040:0.0:1788572306.810182:0:2382:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada60624700 x1875449616089856/t0(0) o9->lustre-OST0001-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572316 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572306.810190:0:2382:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788572306.810194:0:2382:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:0.0:1788572306.810197:0:2382:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada5176a5d8. 00000100:00000200:0.0:1788572306.810201:0:2382:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1875449616089856, offset 0 00000400:00000200:0.0:1788572306.810209:0:2382:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:0.0:1788572306.810221:0:2382:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:0.0:1788572306.810244:0:2382:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:0.0:1788572306.810251:0:2382:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788572306.810257:0:2382:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:0.0:1788572306.810271:0:2382:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788572306.810276:0:2382:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:0.0:1788572306.810281:0:2382:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22268:22268:256:4294967295] 192.168.202.113@tcp LPNI seq info [22264:22264:8:4294967295] 00000400:00000200:0.0:1788572306.810456:0:2382:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:0.0:1788572306.810470:0:2382:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:0.0:1788572306.810477:0:2382:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:0.0:1788572306.810483:0:2382:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada5074a600. 00000800:00000200:0.0:1788572306.810491:0:2382:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:0.0:1788572306.810500:0:2382:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:0.0:1788572306.810505:0:2382:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada5074a600 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788572306.810518:0:2382:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572306.810523:0:2382:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0001-osc-ffff9ada58b0a000: using a timeout of 18091 usecs (19 jiffies) 00000400:00000200:0.0:1788572306.810527:0:2382:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:0.0:1788572306.810530:0:2382:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0001-osc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298891798 00000100:00000001:0.0:1788572306.810533:0:2382:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0001-osc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298897228 00000100:00000001:0.0:1788572306.810536:0:2382:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0001-osc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298891798 00000100:00000001:0.0:1788572306.810538:0:2382:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0001-osc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298897228 00000100:00000001:0.0:1788572306.810540:0:2382:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788572306.810543:0:2382:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572306.810547:0:2382:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572306.810549:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572306.810572:0:77466:0:(import.c:2151:ptlrpc_disconnect_and_idle_import()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572306.810575:0:77466:0:(pinger.c:123:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572306.810581:0:77466:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (4229) 00000100:00000001:2.0:1788572306.810603:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788572306.810606:0:2386:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788572306.810608:0:2386:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788572306.810615:0:2386:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada60624380 x1875449616089472/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.810622:0:2386:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788572306.810623:0:2386:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572306.810626:0:2386:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788572306.810628:0:2386:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788572306.810629:0:2386:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788572306.810632:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616089472:192.168.202.113@tcp:400:kworker.0 00000100:00000001:2.0:1788572306.810635:0:2386:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788572306.810638:0:2386:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788572306.810640:0:2386:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572306.810641:0:2386:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788572306.810645:0:2386:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada45574200. 02000000:00000001:2.0:1788572306.810646:0:2386:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788572306.810650:0:2386:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:2.0:1788572306.810653:0:2386:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada43d96a50. 00000400:00000010:2.0:1788572306.810657:0:2386:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada43c8f110. 00000100:00000200:2.0:1788572306.810661:0:2386:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616089472, portal 10 00000100:00000001:2.0:1788572306.810663:0:2386:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788572306.810664:0:2386:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632861443441536 : -111212266110080 : ffff9ada60624380) 00000100:00000040:2.0:1788572306.810670:0:2386:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada60624380 x1875449616089472/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572322 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.810676:0:2386:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788572306.810679:0:2386:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:2.0:1788572306.810681:0:2386:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada43c8f220. 00000100:00000200:2.0:1788572306.810683:0:2386:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875449616089472, offset 0 00000400:00000200:2.0:1788572306.810688:0:2386:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:2.0:1788572306.810692:0:2386:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572306.810696:0:2386:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:2.0:1788572306.810698:0:2386:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788572306.810701:0:2386:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:2.0:1788572306.810703:0:2386:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788572306.810705:0:2386:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:2.0:1788572306.810708:0:2386:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22269:22269:255:4294967295] 192.168.202.113@tcp LPNI seq info [22265:22265:7:4294967295] 00000400:00000200:2.0:1788572306.810712:0:2386:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572306.810718:0:2386:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:2.0:1788572306.810721:0:2386:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:2.0:1788572306.810726:0:2386:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada535b2300. 00000800:00000200:2.0:1788572306.810729:0:2386:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:2.0:1788572306.810734:0:2386:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:2.0:1788572306.810736:0:2386:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada535b2300 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788572306.810739:0:2386:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572306.810742:0:2386:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: using a timeout of 41273 usecs (42 jiffies) 00000400:00000200:2.0:1788572306.810745:0:2386:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:2.0:1788572306.810747:0:2386:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298891821 00000100:00000001:2.0:1788572306.810749:0:2386:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298897228 00000100:00000001:2.0:1788572306.810751:0:2386:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298891821 00000100:00000001:2.0:1788572306.810753:0:2386:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298897228 00000100:00000001:2.0:1788572306.810755:0:2386:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788572306.810757:0:2386:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572306.810759:0:2386:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788572306.810763:0:2386:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada60627100 x1875449616089728/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.810770:0:2386:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788572306.810771:0:2386:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572306.810773:0:2386:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788572306.810775:0:2386:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788572306.810776:0:2386:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788572306.810779:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616089728:192.168.202.113@tcp:400:kworker.0 00000100:00000001:2.0:1788572306.810781:0:2386:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788572306.810785:0:2386:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788572306.810786:0:2386:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572306.810787:0:2386:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788572306.810791:0:2386:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada45575c00. 02000000:00000001:2.0:1788572306.810792:0:2386:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788572306.810796:0:2386:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:2.0:1788572306.812411:0:2386:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada43d96000. 00000400:00000010:2.0:1788572306.812442:0:2386:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada43c8f4c8. 00000100:00000200:2.0:1788572306.812450:0:2386:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616089728, portal 4 00000100:00000001:2.0:1788572306.812453:0:2386:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788572306.812454:0:2386:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632861443453184 : -111212266098432 : ffff9ada60627100) 00000100:00000040:2.0:1788572306.812472:0:2386:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada60627100 x1875449616089728/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572322 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.812481:0:2386:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788572306.812484:0:2386:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:2.0:1788572306.812487:0:2386:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada43c8f440. 00000100:00000200:2.0:1788572306.812491:0:2386:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1875449616089728, offset 0 00000400:00000200:2.0:1788572306.812497:0:2386:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:2.0:1788572306.812510:0:2386:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572306.812518:0:2386:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:2.0:1788572306.812524:0:2386:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788572306.812528:0:2386:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:2.0:1788572306.812532:0:2386:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788572306.812535:0:2386:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:2.0:1788572306.812539:0:2386:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22270:22270:254:4294967295] 192.168.202.113@tcp LPNI seq info [22266:22266:6:4294967295] 00000400:00000200:2.0:1788572306.812543:0:2386:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572306.812552:0:2386:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:2.0:1788572306.812558:0:2386:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:2.0:1788572306.812567:0:2386:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada617e8500. 00000800:00000200:2.0:1788572306.812573:0:2386:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:2.0:1788572306.812582:0:2386:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:2.0:1788572306.812587:0:2386:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada617e8500 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788572306.812593:0:2386:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572306.812598:0:2386:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: using a timeout of 18796 usecs (19 jiffies) 00000400:00000200:2.0:1788572306.812601:0:2386:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:2.0:1788572306.812605:0:2386:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298891800 00000100:00000001:2.0:1788572306.812608:0:2386:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298897228 00000100:00000001:2.0:1788572306.812611:0:2386:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298891800 00000100:00000001:2.0:1788572306.812613:0:2386:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298897228 00000100:00000001:2.0:1788572306.812616:0:2386:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788572306.812619:0:2386:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572306.812624:0:2386:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572306.812627:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572306.812636:0:2386:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788572306.812638:0:2386:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788572306.812641:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788572306.812643:0:2386:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788572306.812645:0:2386:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788572306.812649:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00000001:2.0:1788572306.812651:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572306.812652:0:2386:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788572306.812654:0:2386:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572306.812656:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788572306.812656:0:2386:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788572306.812658:0:2386:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788572306.812663:0:2386:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada60626680 x1875449616089344/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.812671:0:2386:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788572306.812673:0:2386:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572306.812677:0:2386:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788572306.812680:0:2386:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788572306.812683:0:2386:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788572306.812686:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449616089344:192.168.202.113@tcp:400:kworker.0 00000100:00000001:2.0:1788572306.812691:0:2386:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788572306.812697:0:2386:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788572306.812699:0:2386:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572306.812702:0:2386:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788572306.812707:0:2386:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada45575000. 02000000:00000001:2.0:1788572306.812709:0:2386:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788572306.812713:0:2386:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:2.0:1788572306.812716:0:2386:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada43d96688. 00000400:00000010:2.0:1788572306.812720:0:2386:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada43c8f330. 00000100:00000200:2.0:1788572306.812723:0:2386:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616089344, portal 25 00000100:00000001:2.0:1788572306.812725:0:2386:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788572306.812726:0:2386:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632861443450496 : -111212266101120 : ffff9ada60626680) 00000100:00000040:2.0:1788572306.812731:0:2386:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada60626680 x1875449616089344/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572322 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.812738:0:2386:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788572306.812740:0:2386:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:2.0:1788572306.812742:0:2386:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada43c8fd48. 00000100:00000200:2.0:1788572306.812745:0:2386:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875449616089344, offset 0 00000400:00000200:2.0:1788572306.812749:0:2386:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:2.0:1788572306.812752:0:2386:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572306.812755:0:2386:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:2.0:1788572306.812758:0:2386:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788572306.812760:0:2386:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:2.0:1788572306.812762:0:2386:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788572306.812765:0:2386:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:2.0:1788572306.812768:0:2386:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22271:22271:253:4294967295] 192.168.202.113@tcp LPNI seq info [22267:22267:5:4294967295] 00000400:00000200:2.0:1788572306.812772:0:2386:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572306.812777:0:2386:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:2.0:1788572306.812781:0:2386:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:2.0:1788572306.812784:0:2386:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada617e8f00. 00000800:00000200:2.0:1788572306.812788:0:2386:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:2.0:1788572306.812792:0:2386:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:2.0:1788572306.812795:0:2386:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada617e8f00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788572306.813088:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788572306.813094:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788572306.813182:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788572306.813187:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada5074a600. 00000400:00000200:3.0:1788572306.813197:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572306.813203:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:3.0:1788572306.813209:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada5176a5d8 00000400:00000010:3.0:1788572306.813212:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada5176a5d8. 00000100:00000001:3.0:1788572306.813218:0:2377:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:3.0:1788572306.813224:0:2377:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada60624700 x1875449616089856/t0(0) o9->lustre-OST0001-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572316 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572306.813247:0:2377:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572306.813251:0:2377:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada60624700 x1875449616089856/t0(0) o9->lustre-OST0001-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572316 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572306.813262:0:2377:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572306.813264:0:2377:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000200:3.0:1788572306.813280:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788572306.813282:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada535b2300. 00000400:00000200:3.0:1788572306.813295:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572306.813299:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:3.0:1788572306.813305:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada43c8f220 00000400:00000010:3.0:1788572306.813307:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada43c8f220. 00000100:00000001:3.0:1788572306.813310:0:2377:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:3.0:1788572306.813314:0:2377:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada60624380 x1875449616089472/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572322 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572306.813321:0:2377:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572306.813324:0:2377:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada60624380 x1875449616089472/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572322 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572306.813330:0:2377:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572306.813332:0:2377:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000200:3.0:1788572306.813419:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788572306.813421:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada617e8500. 00000400:00000200:3.0:1788572306.813426:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572306.813429:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:3.0:1788572306.813431:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada43c8f440 00000400:00000010:3.0:1788572306.813433:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada43c8f440. 00000100:00000001:3.0:1788572306.813435:0:2377:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:3.0:1788572306.813440:0:2377:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada60627100 x1875449616089728/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572322 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572306.813447:0:2377:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572306.813450:0:2377:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada60627100 x1875449616089728/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572322 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572306.813455:0:2377:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572306.813457:0:2377:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000001:3.0:1788572306.815166:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572306.815188:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572306.815193:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572306.815195:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572306.815204:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572306.815211:0:2378:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572306.815222:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572306.815227:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572306.815232:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572306.815237:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 10 MB=0x6a9b624294180 00000400:00000200:3.0:1788572306.815242:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572306.815249:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.113@tcp of length 224/224 into md 0x35665 [1] + 224 00000800:00000001:3.0:1788572306.815258:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572306.815262:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572306.815265:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572306.815269:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572306.815279:0:2378:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572306.815298:0:2378:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada60624380 x1875449616089472/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572322 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572306.815310:0:2378:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada60624380 x1875449616089472/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572322 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572306.815319:0:2378:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572306.815328:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572306.815332:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572306.815334:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572306.815336:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572306.815339:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572306.815342:0:2378:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572306.815346:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572306.815349:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572306.815351:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572306.815354:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 4 MB=0x6a9b624294280 00000400:00000200:3.0:1788572306.815357:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572306.815361:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.202.113@tcp of length 224/224 into md 0x35675 [1] + 224 00000800:00000001:3.0:1788572306.815364:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572306.815368:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572306.815369:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572306.815372:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572306.815375:0:2378:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572306.815379:0:2378:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada60627100 x1875449616089728/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572322 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572306.815386:0:2378:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada60627100 x1875449616089728/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572322 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572306.815393:0:2378:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572306.815396:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572306.815398:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572306.815400:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572306.815402:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572306.815405:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572306.815408:0:2378:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572306.815411:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572306.815413:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572306.815416:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572306.815418:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 4 MB=0x6a9b624294300 00000400:00000200:3.0:1788572306.815420:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572306.815424:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.202.113@tcp of length 224/224 into md 0x3565d [1] + 224 00000800:00000001:3.0:1788572306.815427:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572306.815432:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572306.815434:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572306.815436:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572306.815439:0:2378:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572306.815443:0:2378:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada60624700 x1875449616089856/t0(0) o9->lustre-OST0001-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572316 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572306.815450:0:2378:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada60624700 x1875449616089856/t0(0) o9->lustre-OST0001-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572316 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.815757:0:2386:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788572306.815765:0:2386:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:2.0:1788572306.815769:0:2386:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS new deadline estimation for cpu 2 is 4298902356 00000100:00000001:2.0:1788572306.815777:0:2386:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4298897228 00000100:00000001:2.0:1788572306.815784:0:2386:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS new deadline estimation for cpu 3 is 4298902356 00000100:00000001:2.0:1788572306.815787:0:2386:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4298897228 00000100:00000001:2.0:1788572306.815790:0:2386:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788572306.815794:0:2386:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572306.815809:0:2386:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788572306.815816:0:2386:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada60626a00 x1875449616089600/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.815825:0:2386:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788572306.815828:0:2386:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572306.815832:0:2386:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788572306.815836:0:2386:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788572306.815839:0:2386:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788572306.815842:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616089600:192.168.202.113@tcp:400:kworker.0 00000100:00000001:2.0:1788572306.815847:0:2386:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788572306.815890:0:2386:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788572306.815893:0:2386:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572306.815896:0:2386:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788572306.815903:0:2386:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada45575800. 02000000:00000001:2.0:1788572306.815906:0:2386:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788572306.815910:0:2386:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:2.0:1788572306.815914:0:2386:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada43d967e8. 00000400:00000010:2.0:1788572306.815920:0:2386:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada43c8f880. 00000100:00000200:2.0:1788572306.815928:0:2386:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616089600, portal 10 00000100:00000001:2.0:1788572306.815930:0:2386:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788572306.815931:0:2386:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632861443451392 : -111212266100224 : ffff9ada60626a00) 00000100:00000040:2.0:1788572306.815940:0:2386:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada60626a00 x1875449616089600/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572322 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.815947:0:2386:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788572306.815950:0:2386:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:2.0:1788572306.815952:0:2386:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada43c8f7f8. 00000100:00000200:2.0:1788572306.815955:0:2386:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875449616089600, offset 0 00000400:00000200:2.0:1788572306.815962:0:2386:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:2.0:1788572306.815968:0:2386:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572306.815976:0:2386:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:2.0:1788572306.815982:0:2386:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788572306.815987:0:2386:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:2.0:1788572306.815991:0:2386:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788572306.815994:0:2386:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:2.0:1788572306.815998:0:2386:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22272:22272:255:4294967295] 192.168.202.113@tcp LPNI seq info [22268:22268:7:4294967295] 00000400:00000200:2.0:1788572306.816003:0:2386:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572306.816014:0:2386:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:2.0:1788572306.816021:0:2386:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:2.0:1788572306.816027:0:2386:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada617e8500. 00000800:00000200:2.0:1788572306.816033:0:2386:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:2.0:1788572306.816053:0:2386:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:2.0:1788572306.816058:0:2386:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada617e8500 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788572306.816061:0:2386:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572306.816066:0:2386:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: using a timeout of 16452 usecs (17 jiffies) 00000400:00000200:2.0:1788572306.816069:0:2386:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:2.0:1788572306.816071:0:2386:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298891802 00000100:00000001:2.0:1788572306.816073:0:2386:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298902356 00000100:00000001:2.0:1788572306.816075:0:2386:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298891802 00000100:00000001:2.0:1788572306.816077:0:2386:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298902356 00000100:00000001:2.0:1788572306.816079:0:2386:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788572306.816081:0:2386:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:2.0:1788572306.816087:0:2386:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788572306.816089:0:2386:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788572306.816092:0:2386:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada43d96a50. 00000400:00000200:2.0:1788572306.816097:0:2386:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada43c8f110 00000400:00000010:2.0:1788572306.816100:0:2386:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada43c8f110. 00000100:00000001:2.0:1788572306.816102:0:2386:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:2.0:1788572306.816106:0:2386:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada60624380 x1875449616089472/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572322 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788572306.816113:0:2386:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada60624380 x1875449616089472/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572322 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.816120:0:2386:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788572306.816122:0:2386:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572306.816124:0:2386:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:2.0:1788572306.816128:0:2386:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788572306.816130:0:2386:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788572306.816133:0:2386:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572306.816135:0:2386:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788572306.816144:0:2386:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada51140460 time=151 v=5 (5 5 5 5) 00000100:00001000:2.0:1788572306.816149:0:2386:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada51140418 time=97 v=5 (2 0 0 0) 00000100:00000001:2.0:1788572306.816152:0:2386:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788572306.816153:0:2386:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572306.816157:0:2386:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788572306.816161:0:2386:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572306.816165:0:2386:0:(client.c:3110:ptlrpc_free_committed()) Process entered 00000100:00100000:2.0:1788572306.816167:0:2386:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 219043334508 gen 9 00000100:00000040:2.0:1788572306.816173:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60546a00 x1875449615935488/t219043334109(219043334109) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.816181:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.816184:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60546a00 x1875449615935488/t219043334109(219043334109) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.816191:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.816194:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.816196:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada42a28800. 02000000:00000001:2.0:1788572306.816201:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.816204:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.816206:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=403 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.816209:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.816212:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada4551d800. 02000000:00000001:2.0:1788572306.816215:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.816240:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.816242:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60546a00. 00000100:00000001:2.0:1788572306.816246:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.816248:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.816253:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada51789f80 x1875449615935872/t219043334110(219043334110) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.816260:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.816263:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada51789f80 x1875449615935872/t219043334110(219043334110) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.816269:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.816270:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.816272:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4546cc00. 02000000:00000001:2.0:1788572306.816274:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.816275:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.816276:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=402 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.816278:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.816280:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454ade00. 02000000:00000001:2.0:1788572306.816282:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.816487:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.816489:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada51789f80. 00000100:00000001:2.0:1788572306.816493:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.816495:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.816501:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada51788e00 x1875449615936256/t219043334111(219043334111) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.816508:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.816511:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada51788e00 x1875449615936256/t219043334111(219043334111) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.816517:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.816518:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.816521:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4546c400. 02000000:00000001:2.0:1788572306.816523:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.816525:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.816526:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=401 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.816528:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.816530:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454adc00. 02000000:00000001:2.0:1788572306.816531:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.816532:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.816534:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada51788e00. 00000100:00000001:2.0:1788572306.816536:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.816537:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.816542:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada5178aa00 x1875449615936640/t219043334112(219043334112) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.816548:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.816551:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada5178aa00 x1875449615936640/t219043334112(219043334112) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.816557:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.816559:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.816560:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4546e800. 02000000:00000001:2.0:1788572306.816563:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.816564:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.816566:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=400 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.816568:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.816569:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60b9ee00. 02000000:00000001:2.0:1788572306.816572:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.816573:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.816574:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada5178aa00. 00000100:00000001:2.0:1788572306.816577:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.816578:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.816583:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada5178a680 x1875449615937024/t219043334113(219043334113) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.816590:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.816593:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada5178a680 x1875449615937024/t219043334113(219043334113) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.816605:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.816606:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.816608:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4546e400. 02000000:00000001:2.0:1788572306.816610:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.816611:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.816612:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=399 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.816613:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.816614:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60b9e600. 02000000:00000001:2.0:1788572306.816616:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.816617:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.816618:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada5178a680. 00000100:00000001:2.0:1788572306.816620:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.816621:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.816625:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada51788380 x1875449615937408/t219043334114(219043334114) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.816631:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.816634:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada51788380 x1875449615937408/t219043334114(219043334114) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.816640:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.816641:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.816642:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4546d000. 02000000:00000001:2.0:1788572306.816645:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.816646:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.816647:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=398 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.816648:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.816650:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60b9f400. 02000000:00000001:2.0:1788572306.816652:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.816653:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.816655:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada51788380. 00000100:00000001:2.0:1788572306.817032:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.817034:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.817052:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada51789500 x1875449615937792/t219043334115(219043334115) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.817064:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.817067:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada51789500 x1875449615937792/t219043334115(219043334115) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.817073:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.817075:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.817077:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4546dc00. 02000000:00000001:2.0:1788572306.817080:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.817081:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.817082:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=397 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.817084:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.817086:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60b9f200. 02000000:00000001:2.0:1788572306.817088:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.817089:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.817091:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada51789500. 00000100:00000001:2.0:1788572306.817093:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.817095:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.817100:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada5178b100 x1875449615938176/t219043334116(219043334116) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.817105:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.817108:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada5178b100 x1875449615938176/t219043334116(219043334116) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.817113:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.817114:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.817115:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4546c000. 02000000:00000001:2.0:1788572306.817117:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.817118:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.817119:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=396 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.817120:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.817121:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60b9ec00. 02000000:00000001:2.0:1788572306.817123:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.817124:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.817126:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada5178b100. 00000100:00000001:2.0:1788572306.817128:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.817129:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.817132:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada51788700 x1875449615938560/t219043334117(219043334117) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.817138:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.817140:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada51788700 x1875449615938560/t219043334117(219043334117) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.817145:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.817147:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.817148:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4546d400. 02000000:00000001:2.0:1788572306.817150:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.817151:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.817153:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=395 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.817155:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.817156:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60b9f600. 02000000:00000001:2.0:1788572306.817158:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.817159:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.817160:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada51788700. 00000100:00000001:2.0:1788572306.817162:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.817163:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.817169:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada51789180 x1875449615938944/t219043334118(219043334118) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.817175:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.817178:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada51789180 x1875449615938944/t219043334118(219043334118) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.817183:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.817185:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.817186:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4546f800. 02000000:00000001:2.0:1788572306.817189:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.817190:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.817191:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=394 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.817193:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.817195:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60b9f000. 02000000:00000001:2.0:1788572306.817197:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.817198:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.817199:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada51789180. 00000100:00000001:2.0:1788572306.817202:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.817203:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.817208:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada51789880 x1875449615939328/t219043334119(219043334119) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.817215:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.817218:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada51789880 x1875449615939328/t219043334119(219043334119) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.817224:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.819812:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.819822:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4546f000. 02000000:00000001:2.0:1788572306.819835:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.819839:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.819841:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=393 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.819845:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.819848:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60b9e000. 02000000:00000001:2.0:1788572306.819878:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.819881:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.819884:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada51789880. 00000100:00000001:2.0:1788572306.819889:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.819891:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.819907:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada5178b480 x1875449615939712/t219043334120(219043334120) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.819917:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.819920:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada5178b480 x1875449615939712/t219043334120(219043334120) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.819927:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.819928:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.819930:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4546c800. 02000000:00000001:2.0:1788572306.819932:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.819935:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.819936:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=392 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.819937:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.819939:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60b9fc00. 02000000:00000001:2.0:1788572306.819941:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.819942:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.819943:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada5178b480. 00000100:00000001:2.0:1788572306.819945:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.819946:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.819950:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada5178ad80 x1875449615940096/t219043334121(219043334121) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.819957:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.819960:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada5178ad80 x1875449615940096/t219043334121(219043334121) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.819966:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.819967:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.819969:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4546ec00. 02000000:00000001:2.0:1788572306.819971:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.819972:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.819973:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=391 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.819975:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.819976:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60b9e200. 02000000:00000001:2.0:1788572306.819978:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.819979:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.819980:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada5178ad80. 00000100:00000001:2.0:1788572306.819982:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.819984:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.819989:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada51788a80 x1875449615940480/t219043334122(219043334122) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.819995:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.819998:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada51788a80 x1875449615940480/t219043334122(219043334122) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.820004:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.820005:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.820007:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4546e000. 02000000:00000001:2.0:1788572306.820009:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.820010:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.820012:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=390 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.820013:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.820015:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60b9fa00. 02000000:00000001:2.0:1788572306.820017:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.820018:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.820019:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada51788a80. 00000100:00000001:2.0:1788572306.820022:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.820023:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.820029:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60547b80 x1875449615940864/t219043334123(219043334123) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.820035:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.820049:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60547b80 x1875449615940864/t219043334123(219043334123) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.820055:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.820056:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.820058:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada42a28400. 02000000:00000001:2.0:1788572306.820061:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.820062:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.820063:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=389 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.820065:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.820067:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada4551da00. 02000000:00000001:2.0:1788572306.820069:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.820070:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.820071:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60547b80. 00000100:00000001:2.0:1788572306.820074:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.820075:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.820080:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45537100 x1875449615941248/t219043334124(219043334124) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.820087:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.820089:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45537100 x1875449615941248/t219043334124(219043334124) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.820096:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.820097:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.820099:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada454a5400. 02000000:00000001:2.0:1788572306.820103:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.820104:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.820105:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=388 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.820107:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.820109:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada48902600. 02000000:00000001:2.0:1788572306.820111:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.820112:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.820114:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45537100. 00000100:00000001:2.0:1788572306.820117:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.820118:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.820123:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45535500 x1875449615941632/t219043334125(219043334125) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.820130:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.820133:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45535500 x1875449615941632/t219043334125(219043334125) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.820138:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.820139:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.820141:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada454a7000. 02000000:00000001:2.0:1788572306.820143:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.820144:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.820145:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=387 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.820147:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.820148:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada48903a00. 02000000:00000001:2.0:1788572306.820150:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.820151:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.820152:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45535500. 00000100:00000001:2.0:1788572306.820155:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.820156:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.820160:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45534380 x1875449615942016/t219043334126(219043334126) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.820171:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.820173:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45534380 x1875449615942016/t219043334126(219043334126) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.820178:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.820180:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.820181:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada454a4800. 02000000:00000001:2.0:1788572306.820183:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.820184:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.820185:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=386 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.820187:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.820188:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada48902200. 02000000:00000001:2.0:1788572306.820189:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.820190:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.820191:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45534380. 00000100:00000001:2.0:1788572306.820194:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.820195:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.820200:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada51788000 x1875449615942400/t219043334127(219043334127) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.820205:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.820207:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada51788000 x1875449615942400/t219043334127(219043334127) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.820211:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.820212:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.820213:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4546d800. 02000000:00000001:2.0:1788572306.820216:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.820217:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.820218:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=385 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.820219:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.820221:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60b9f800. 00000100:00000001:0.0:1788572306.823774:0:2382:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788572306.823783:0:2382:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=5 : 5 : 5) 00000100:00000001:0.0:1788572306.823791:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788572306.823794:0:2382:0:(client.c:1991:ptlrpc_check_set()) Process entered 02000000:00000001:2.0:1788572306.823819:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.823825:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.823831:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada51788000. 00000100:00000001:2.0:1788572306.823842:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.823843:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.823884:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada51789c00 x1875449615942784/t219043334128(219043334128) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.823895:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.823898:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada51789c00 x1875449615942784/t219043334128(219043334128) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.823906:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.823908:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.823912:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4546fc00. 02000000:00000001:2.0:1788572306.823917:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.823920:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.823923:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=384 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.823926:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.823929:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454ad000. 02000000:00000001:2.0:1788572306.823933:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.823934:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.823935:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada51789c00. 00000100:00000001:2.0:1788572306.823938:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.823939:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.823944:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada5178bb80 x1875449615943168/t219043334129(219043334129) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.823951:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.823954:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada5178bb80 x1875449615943168/t219043334129(219043334129) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.823960:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.823961:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.823963:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4546f400. 02000000:00000001:2.0:1788572306.823965:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.823966:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.823967:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=383 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.823969:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.823971:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454ad600. 02000000:00000001:2.0:1788572306.823973:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.823974:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.823975:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada5178bb80. 00000100:00000001:2.0:1788572306.823978:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.823979:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.823984:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada5178a300 x1875449615943552/t219043334130(219043334130) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.823990:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.823993:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada5178a300 x1875449615943552/t219043334130(219043334130) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.823998:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.824000:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.824001:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada51734400. 02000000:00000001:2.0:1788572306.824005:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.824006:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.824007:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=382 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.824009:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.824011:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454ac000. 02000000:00000001:2.0:1788572306.824013:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.824014:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.824015:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada5178a300. 00000100:00000001:2.0:1788572306.824017:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.824019:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.824023:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada5178b800 x1875449615943936/t219043334131(219043334131) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.824031:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.824034:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada5178b800 x1875449615943936/t219043334131(219043334131) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.824049:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.824051:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.824052:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada51736c00. 02000000:00000001:2.0:1788572306.824055:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.824057:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.824058:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=381 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.824060:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.824061:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454ad800. 02000000:00000001:2.0:1788572306.824063:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.824064:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.824066:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada5178b800. 00000100:00000001:2.0:1788572306.824068:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.824069:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.824074:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60a49500 x1875449615944320/t219043334132(219043334132) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.824080:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.824083:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60a49500 x1875449615944320/t219043334132(219043334132) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.824089:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.824090:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.824092:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada51736800. 02000000:00000001:2.0:1788572306.824094:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.824095:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.824096:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=380 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.824098:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.824099:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454ad200. 02000000:00000001:2.0:1788572306.824101:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.824102:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.824103:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60a49500. 00000100:00000001:2.0:1788572306.824106:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.824107:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.824112:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60a4aa00 x1875449615944704/t219043334133(219043334133) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.824119:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.824121:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60a4aa00 x1875449615944704/t219043334133(219043334133) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.824127:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.824129:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.824131:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada51735c00. 02000000:00000001:2.0:1788572306.824133:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.824135:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.824136:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=379 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.824138:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.824139:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454ac800. 02000000:00000001:2.0:1788572306.824141:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.824142:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.824143:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60a4aa00. 00000100:00000001:2.0:1788572306.824146:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.824147:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.824152:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60a4a680 x1875449615945088/t219043334134(219043334134) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.824158:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.824161:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60a4a680 x1875449615945088/t219043334134(219043334134) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.824167:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.824169:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.824170:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada51737c00. 02000000:00000001:2.0:1788572306.824173:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.824174:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.824175:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=378 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.824176:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.824178:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454aca00. 02000000:00000001:2.0:1788572306.824180:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.824181:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.824183:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60a4a680. 00000100:00000001:2.0:1788572306.824185:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.824186:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.824191:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60a48380 x1875449615945472/t219043334135(219043334135) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.824198:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.824201:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60a48380 x1875449615945472/t219043334135(219043334135) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.824208:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.824209:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.824211:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada51734800. 02000000:00000001:2.0:1788572306.824213:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.824214:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.824215:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=377 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.824217:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.824218:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454ac600. 02000000:00000001:2.0:1788572306.824221:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.824222:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.824223:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60a48380. 00000100:00000001:2.0:1788572306.824238:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.824240:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.824245:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60a48e00 x1875449615945856/t219043334136(219043334136) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.824252:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.824255:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60a48e00 x1875449615945856/t219043334136(219043334136) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.824262:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.824263:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.824265:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada51736000. 02000000:00000001:2.0:1788572306.824267:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.824268:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.824270:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=376 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.824272:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.824273:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454ad400. 02000000:00000001:2.0:1788572306.824275:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.824276:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.824278:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60a48e00. 00000100:00000001:2.0:1788572306.824280:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.824281:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.826426:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45536a00 x1875449615946240/t219043334137(219043334137) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.826451:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.826458:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45536a00 x1875449615946240/t219043334137(219043334137) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.826466:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.826469:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.826474:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada454a7400. 02000000:00000001:2.0:1788572306.826482:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.826486:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.826487:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=375 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.826490:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.826493:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada48903200. 02000000:00000001:2.0:1788572306.826498:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.826499:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.826502:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45536a00. 00000100:00000001:2.0:1788572306.826506:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.826508:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.826514:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45534e00 x1875449615946624/t219043334138(219043334138) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.826521:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.826524:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45534e00 x1875449615946624/t219043334138(219043334138) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.826530:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.826531:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.826533:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada454a6c00. 02000000:00000001:2.0:1788572306.826535:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.826537:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.826538:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=374 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.826540:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.826541:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada48902800. 02000000:00000001:2.0:1788572306.826544:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.826545:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.826547:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45534e00. 00000100:00000001:2.0:1788572306.826551:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.826552:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.826556:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45535f80 x1875449615947008/t219043334139(219043334139) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.826562:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.826564:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45535f80 x1875449615947008/t219043334139(219043334139) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.826570:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.826571:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.826573:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada454a4000. 02000000:00000001:2.0:1788572306.826575:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.826576:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.826577:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=373 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.826579:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.826580:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada48903c00. 02000000:00000001:2.0:1788572306.826582:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.826583:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.826585:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45535f80. 00000100:00000001:2.0:1788572306.826587:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.826588:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.826593:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45534a80 x1875449615947392/t219043334140(219043334140) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.826600:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.826602:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45534a80 x1875449615947392/t219043334140(219043334140) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.826608:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.826609:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.826610:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada454a5800. 02000000:00000001:2.0:1788572306.826613:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.826614:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.826615:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=372 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.826617:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.826619:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada48902a00. 02000000:00000001:2.0:1788572306.826620:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.826621:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.826622:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45534a80. 00000100:00000001:2.0:1788572306.826624:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.826625:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.826630:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45537800 x1875449615947776/t219043334141(219043334141) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.826636:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.826638:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45537800 x1875449615947776/t219043334141(219043334141) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.826644:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.826645:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.826647:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada454a7c00. 02000000:00000001:2.0:1788572306.826649:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.826650:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.826651:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=371 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.826653:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.826654:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada48903600. 02000000:00000001:2.0:1788572306.826725:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.826727:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.826728:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45537800. 00000100:00000001:2.0:1788572306.826731:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.826732:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.826737:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60a49880 x1875449615948160/t219043334142(219043334142) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.826743:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.826747:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60a49880 x1875449615948160/t219043334142(219043334142) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.826752:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.826753:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.826755:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada51737000. 02000000:00000001:2.0:1788572306.826757:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.826759:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.826760:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=370 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.826762:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.826763:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454ada00. 02000000:00000001:2.0:1788572306.826764:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.826766:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.826767:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60a49880. 00000100:00000001:2.0:1788572306.826769:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.826770:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.826774:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60a49f80 x1875449615948544/t219043334143(219043334143) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.826779:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.826782:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60a49f80 x1875449615948544/t219043334143(219043334143) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.826787:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.826788:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.826789:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada51737400. 02000000:00000001:2.0:1788572306.826791:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.826791:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.826792:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=369 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.826794:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.826795:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada6163ba00. 02000000:00000001:2.0:1788572306.826912:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.826913:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.826914:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60a49f80. 00000100:00000001:2.0:1788572306.826917:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.826918:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.826924:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60a49180 x1875449615948928/t219043334144(219043334144) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.826931:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.826933:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60a49180 x1875449615948928/t219043334144(219043334144) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.826939:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.826942:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.826943:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada51735800. 02000000:00000001:2.0:1788572306.826946:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.826947:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.826949:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=368 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.826951:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.826952:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada5306be00. 02000000:00000001:2.0:1788572306.826955:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.826957:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.826958:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60a49180. 00000100:00000001:2.0:1788572306.826960:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.826961:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.826966:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60a49c00 x1875449615949312/t219043334145(219043334145) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.826973:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.826975:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60a49c00 x1875449615949312/t219043334145(219043334145) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.826981:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.826983:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.826984:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada51737800. 02000000:00000001:2.0:1788572306.826986:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.826988:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.826989:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=367 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.826991:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.826993:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada509c5000. 02000000:00000001:2.0:1788572306.826995:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.826996:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.826998:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60a49c00. 00000100:00000001:2.0:1788572306.827000:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.827001:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.827007:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60a4b480 x1875449615949696/t219043334146(219043334146) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.827013:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.827016:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60a4b480 x1875449615949696/t219043334146(219043334146) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.827023:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.827025:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.827027:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada51734c00. 02000000:00000001:2.0:1788572306.827029:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.827030:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.827031:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=366 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.827033:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.827035:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada42e2e400. 02000000:00000001:2.0:1788572306.827067:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.827069:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.827071:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60a4b480. 00000100:00000001:2.0:1788572306.827074:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.827075:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.827081:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60a48000 x1875449615950080/t219043334147(219043334147) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.827087:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.827090:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60a48000 x1875449615950080/t219043334147(219043334147) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.827097:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.827099:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.827100:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada507d4000. 02000000:00000001:2.0:1788572306.827103:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.827104:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.827106:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=365 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.827108:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.827109:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60b9e400. 02000000:00000001:2.0:1788572306.827111:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.827112:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.827113:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60a48000. 00000100:00000001:2.0:1788572306.827116:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.827117:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.827122:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60a4ad80 x1875449615950464/t219043334148(219043334148) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.827133:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.827137:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60a4ad80 x1875449615950464/t219043334148(219043334148) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.827143:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.827145:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.827146:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada507d5800. 02000000:00000001:2.0:1788572306.827149:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.827150:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.827151:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=364 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.827153:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.827154:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51021e00. 02000000:00000001:2.0:1788572306.827157:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.827158:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.827160:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60a4ad80. 00000100:00000001:2.0:1788572306.827162:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.827163:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.827168:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60a4a300 x1875449615950848/t219043334149(219043334149) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.827174:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.827177:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60a4a300 x1875449615950848/t219043334149(219043334149) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.827183:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.827185:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.827186:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada507d4800. 02000000:00000001:2.0:1788572306.827188:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.827190:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.827191:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=363 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.827193:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.827194:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51021200. 02000000:00000001:2.0:1788572306.827196:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.827197:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.827199:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60a4a300. 00000100:00000001:2.0:1788572306.827201:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.827203:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.827207:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60a4bb80 x1875449615951232/t219043334150(219043334150) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.827214:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.827216:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60a4bb80 x1875449615951232/t219043334150(219043334150) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.827221:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.827223:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.827224:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada507d7000. 02000000:00000001:2.0:1788572306.827226:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.827227:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.827228:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=362 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.827230:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.827232:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51021600. 02000000:00000001:2.0:1788572306.827234:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.827235:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.827236:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60a4bb80. 00000100:00000001:2.0:1788572306.827238:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.827239:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.827244:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada44816680 x1875449615951616/t219043334151(219043334151) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.827251:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.827254:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada44816680 x1875449615951616/t219043334151(219043334151) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.827260:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.827261:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.827263:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada507d6000. 02000000:00000001:2.0:1788572306.827265:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.827267:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.827268:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=361 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.827270:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.827271:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51021400. 02000000:00000001:2.0:1788572306.827273:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.827274:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.827275:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada44816680. 00000100:00000001:2.0:1788572306.827279:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.827280:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788572306.827699:0:2378:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000200:3.0:1788572306.827770:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788572306.827775:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada617e8f00. 00000400:00000200:3.0:1788572306.827782:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572306.827789:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:3.0:1788572306.827796:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada43c8fd48 00000400:00000010:3.0:1788572306.827814:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada43c8fd48. 00000100:00000001:3.0:1788572306.827819:0:2378:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:3.0:1788572306.827826:0:2378:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada60626680 x1875449616089344/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572322 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572306.827838:0:2378:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572306.827841:0:2378:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada60626680 x1875449616089344/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572322 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572306.827869:0:2378:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572306.827871:0:2378:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000001:3.0:1788572306.827879:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572306.827887:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:3.0:1788572306.827959:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788572306.827961:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada617e8500. 00000400:00000200:3.0:1788572306.827965:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572306.827968:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:3.0:1788572306.827972:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada43c8f7f8 00000400:00000010:3.0:1788572306.827974:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada43c8f7f8. 00000100:00000001:3.0:1788572306.827976:0:2378:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:3.0:1788572306.827981:0:2378:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada60626a00 x1875449616089600/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572322 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572306.827989:0:2378:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572306.827991:0:2378:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada60626a00 x1875449616089600/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572322 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572306.827998:0:2378:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572306.828000:0:2378:0:(events.c:69:request_out_callback()) Process leaving 00000100:00000040:2.0:1788572306.829006:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada44815880 x1875449615952000/t219043334152(219043334152) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.829026:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.829035:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada44815880 x1875449615952000/t219043334152(219043334152) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.829050:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.829052:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.829056:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada507d7400. 02000000:00000001:2.0:1788572306.829060:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.829063:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.829065:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=360 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.829067:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.829070:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada42e2fc00. 02000000:00000001:2.0:1788572306.829073:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.829075:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.829077:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada44815880. 00000100:00000001:2.0:1788572306.829080:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.829081:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.829087:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada44815180 x1875449615952384/t219043334153(219043334153) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.829094:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.829096:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada44815180 x1875449615952384/t219043334153(219043334153) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.829102:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.829104:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.829105:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada507d7c00. 02000000:00000001:2.0:1788572306.829107:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.829109:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.829110:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=359 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.829112:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.829113:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada58967a00. 02000000:00000001:2.0:1788572306.829116:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.829117:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.829118:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada44815180. 00000100:00000001:2.0:1788572306.829120:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.829121:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.829126:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada44817480 x1875449615952768/t219043334154(219043334154) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.829131:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.829133:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada44817480 x1875449615952768/t219043334154(219043334154) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572313 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.829139:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.829140:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.829141:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada507d5c00. 02000000:00000001:2.0:1788572306.829143:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.829144:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.829145:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=358 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.829147:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.829148:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51021000. 02000000:00000001:2.0:1788572306.829150:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.829151:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.829152:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada44817480. 00000100:00000001:2.0:1788572306.829154:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.829155:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.829160:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada44814380 x1875449615953152/t219043334155(219043334155) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.829166:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.829168:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada44814380 x1875449615953152/t219043334155(219043334155) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.829174:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.829175:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.829176:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada507d4400. 02000000:00000001:2.0:1788572306.829178:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.829179:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.829180:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=357 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.829182:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.829183:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada438f0c00. 02000000:00000001:2.0:1788572306.829185:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.829186:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.829188:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada44814380. 00000100:00000001:2.0:1788572306.829190:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.829191:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.829196:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada44815500 x1875449615953536/t219043334156(219043334156) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.829202:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.829205:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada44815500 x1875449615953536/t219043334156(219043334156) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.829211:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.829212:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.829214:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada454a5c00. 02000000:00000001:2.0:1788572306.829217:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.829219:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.829220:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=356 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.829222:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.829223:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada488fa600. 02000000:00000001:2.0:1788572306.829730:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.829733:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.829735:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada44815500. 00000100:00000001:2.0:1788572306.829739:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.829740:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.829748:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45534000 x1875449615953920/t219043334157(219043334157) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.829756:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.829759:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45534000 x1875449615953920/t219043334157(219043334157) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.829765:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.829767:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.829769:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada454a4400. 02000000:00000001:2.0:1788572306.829772:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.829773:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.829774:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=355 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.829776:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.829777:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada5170e800. 02000000:00000001:2.0:1788572306.829780:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.829781:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.829782:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45534000. 00000100:00000001:2.0:1788572306.829785:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.829786:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.829791:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45535c00 x1875449615954304/t219043334158(219043334158) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.829918:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.829923:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45535c00 x1875449615954304/t219043334158(219043334158) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.829930:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.829931:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.829933:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada42a29000. 02000000:00000001:2.0:1788572306.829937:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.829938:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.829939:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=354 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.829941:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.829942:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada5170e400. 02000000:00000001:2.0:1788572306.829944:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.829945:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.829946:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45535c00. 00000100:00000001:2.0:1788572306.829950:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.829951:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.829956:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada44817100 x1875449615954688/t219043334159(219043334159) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.829967:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.829969:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada44817100 x1875449615954688/t219043334159(219043334159) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.829975:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.829976:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.829978:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada507d6c00. 02000000:00000001:2.0:1788572306.829980:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.829982:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.829983:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=353 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.829985:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.829986:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada488fb800. 02000000:00000001:2.0:1788572306.829988:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.829989:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.829991:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada44817100. 00000100:00000001:2.0:1788572306.829993:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.829994:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.829998:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada44814700 x1875449615955072/t219043334160(219043334160) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.830004:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.830007:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada44814700 x1875449615955072/t219043334160(219043334160) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.830013:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.830014:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.830015:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada507d7800. 02000000:00000001:2.0:1788572306.830017:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.830018:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.830019:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=352 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.830021:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.830023:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada488fae00. 02000000:00000001:2.0:1788572306.830025:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.830026:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.830028:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada44814700. 00000100:00000001:2.0:1788572306.830030:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.830031:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.830035:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada516ec000 x1875449615955456/t219043334161(219043334161) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.830051:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.830054:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada516ec000 x1875449615955456/t219043334161(219043334161) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.830060:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.830061:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.830062:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada507d5400. 02000000:00000001:2.0:1788572306.830065:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.830066:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.830067:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=351 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.830069:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.830070:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada488fba00. 02000000:00000001:2.0:1788572306.830072:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.830074:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.830075:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada516ec000. 00000100:00000001:2.0:1788572306.830078:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.830080:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.830085:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada516eca80 x1875449615955840/t219043334162(219043334162) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.830091:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.830094:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada516eca80 x1875449615955840/t219043334162(219043334162) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.830100:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.830101:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.830103:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada507d5000. 02000000:00000001:2.0:1788572306.830105:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.830107:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.830108:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=350 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.830110:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.830112:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada488fac00. 02000000:00000001:2.0:1788572306.830114:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.830115:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.830117:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada516eca80. 00000100:00000001:2.0:1788572306.830119:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.830120:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.830124:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada516edf80 x1875449615956224/t219043334163(219043334163) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.830131:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.830133:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada516edf80 x1875449615956224/t219043334163(219043334163) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.830139:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.830140:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.830142:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada507d6400. 02000000:00000001:2.0:1788572306.830143:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.830145:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.830146:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=349 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.830148:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.830149:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada488faa00. 02000000:00000001:2.0:1788572306.830152:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.830153:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.830154:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada516edf80. 00000100:00000001:2.0:1788572306.830159:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.830160:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.830164:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada516ece00 x1875449615956608/t219043334164(219043334164) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.830171:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.830173:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada516ece00 x1875449615956608/t219043334164(219043334164) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.830178:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.830179:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.830180:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada507d6800. 02000000:00000001:2.0:1788572306.830182:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.830184:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.830185:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=348 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.830186:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.830187:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada488fa400. 02000000:00000001:2.0:1788572306.830189:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.830190:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.830191:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada516ece00. 00000100:00000001:2.0:1788572306.830193:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.830194:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.830199:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada516ee680 x1875449615956992/t219043334165(219043334165) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.830205:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.830207:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada516ee680 x1875449615956992/t219043334165(219043334165) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.830212:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.830213:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.830214:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada507d4c00. 02000000:00000001:2.0:1788572306.830216:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.830217:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.830218:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=347 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.830220:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.830222:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60123800. 02000000:00000001:2.0:1788572306.830225:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.832066:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.832072:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada516ee680. 00000100:00000001:2.0:1788572306.832083:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.832085:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.832099:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada516ec380 x1875449615957376/t219043334166(219043334166) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.832109:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.832113:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada516ec380 x1875449615957376/t219043334166(219043334166) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.832120:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.832123:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.832126:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada51751000. 02000000:00000001:2.0:1788572306.832131:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.832134:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.832135:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=346 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.832138:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.832140:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada4120a600. 02000000:00000001:2.0:1788572306.832147:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.832148:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.832150:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada516ec380. 00000100:00000001:2.0:1788572306.832153:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.832154:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.832159:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada516ed500 x1875449615957760/t219043334167(219043334167) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.832166:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.832169:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada516ed500 x1875449615957760/t219043334167(219043334167) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.832176:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.832177:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.832178:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada51751c00. 02000000:00000001:2.0:1788572306.832181:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.832182:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.832183:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=345 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.832185:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.832186:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada43bed000. 02000000:00000001:2.0:1788572306.832188:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.832189:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.832191:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada516ed500. 00000100:00000001:2.0:1788572306.832195:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.832196:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.832200:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada516ef100 x1875449615958144/t219043334168(219043334168) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.832206:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.832209:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada516ef100 x1875449615958144/t219043334168(219043334168) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.832214:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.832215:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.832217:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada51753000. 02000000:00000001:2.0:1788572306.832219:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.832220:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.832221:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=344 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.832223:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.832224:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada518e2a00. 02000000:00000001:2.0:1788572306.832237:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.832238:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.832240:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada516ef100. 00000100:00000001:2.0:1788572306.832243:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.832244:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.832249:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada516ec700 x1875449615958528/t219043334169(219043334169) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.832255:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.832265:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada516ec700 x1875449615958528/t219043334169(219043334169) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.832270:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.832272:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.832273:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada51752c00. 02000000:00000001:2.0:1788572306.832276:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.832277:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.832278:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=343 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.832280:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.832282:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada518e3600. 02000000:00000001:2.0:1788572306.832284:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.832285:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.832286:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada516ec700. 00000100:00000001:2.0:1788572306.832289:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.832290:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.832295:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada516ed180 x1875449615958912/t219043334170(219043334170) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.832306:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.832309:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada516ed180 x1875449615958912/t219043334170(219043334170) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.832315:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.832316:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.832317:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada51750400. 02000000:00000001:2.0:1788572306.832320:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.832321:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.832322:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=342 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.832324:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.832325:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada518e2e00. 02000000:00000001:2.0:1788572306.832327:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.832329:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.832330:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada516ed180. 00000100:00000001:2.0:1788572306.832332:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.832333:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.832336:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada516ed880 x1875449615959296/t219043334171(219043334171) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.832342:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.832346:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada516ed880 x1875449615959296/t219043334171(219043334171) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.832352:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.832353:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.832355:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada51752400. 02000000:00000001:2.0:1788572306.832357:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.832358:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.832359:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=341 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.832361:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.832362:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada518e2200. 02000000:00000001:2.0:1788572306.832364:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.832365:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.832367:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada516ed880. 00000100:00000001:2.0:1788572306.832369:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.832370:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.832374:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada516ef480 x1875449615959680/t219043334172(219043334172) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.832381:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.832383:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada516ef480 x1875449615959680/t219043334172(219043334172) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.832389:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.832390:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.832392:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada51753c00. 02000000:00000001:2.0:1788572306.832394:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.832395:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.832396:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=340 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.832398:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.832399:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada488fa000. 02000000:00000001:2.0:1788572306.832401:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.832402:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.832404:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada516ef480. 00000100:00000001:2.0:1788572306.832405:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.832406:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.832411:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada516eed80 x1875449615960064/t219043334173(219043334173) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.832417:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.832420:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada516eed80 x1875449615960064/t219043334173(219043334173) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.832426:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.832427:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.832429:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada51751800. 02000000:00000001:2.0:1788572306.832430:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.832431:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.832432:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=339 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.832434:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.832436:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada4137ee00. 02000000:00000001:2.0:1788572306.832439:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.832440:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.832441:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada516eed80. 00000100:00000001:2.0:1788572306.832443:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.832445:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.832449:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada516ee300 x1875449615960448/t219043334174(219043334174) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.832455:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.832458:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada516ee300 x1875449615960448/t219043334174(219043334174) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.832463:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.832464:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.832465:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada51750c00. 02000000:00000001:2.0:1788572306.832467:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.832468:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.832469:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=338 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.832471:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.832472:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada511e8a00. 02000000:00000001:2.0:1788572306.832475:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.832476:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.832477:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada516ee300. 00000100:00000001:2.0:1788572306.832479:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.832480:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.832484:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada516efb80 x1875449615960832/t219043334175(219043334175) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.832490:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.832493:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada516efb80 x1875449615960832/t219043334175(219043334175) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.832499:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.832500:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.832501:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada51750800. 02000000:00000001:2.0:1788572306.832503:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.832505:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.832506:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=337 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.832507:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.832508:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada48a60a00. 02000000:00000001:2.0:1788572306.832511:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.832512:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.832513:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada516efb80. 00000100:00000001:2.0:1788572306.832515:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.832516:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.832521:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada516edc00 x1875449615961216/t219043334176(219043334176) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.832527:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.832530:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada516edc00 x1875449615961216/t219043334176(219043334176) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.832536:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.832537:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.832538:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada51750000. 02000000:00000001:2.0:1788572306.832540:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.832541:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.832542:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=336 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.832544:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.832545:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada48a60e00. 02000000:00000001:2.0:1788572306.832547:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.832548:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.832549:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada516edc00. 00000100:00000001:2.0:1788572306.832551:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.832553:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.832557:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada516ef800 x1875449615961600/t219043334177(219043334177) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.832564:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.832567:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada516ef800 x1875449615961600/t219043334177(219043334177) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.832573:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.832575:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.832576:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada51752800. 02000000:00000001:2.0:1788572306.832578:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.832580:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.832581:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=335 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.832583:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.832584:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada518e3800. 02000000:00000001:2.0:1788572306.832585:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.832586:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.832588:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada516ef800. 00000100:00000001:2.0:1788572306.832589:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.832591:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.832596:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada516eea00 x1875449615961984/t219043334178(219043334178) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.832602:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.832605:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada516eea00 x1875449615961984/t219043334178(219043334178) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.832611:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.832612:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.832613:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada51751400. 02000000:00000001:2.0:1788572306.832615:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.832616:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.832617:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=334 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.832620:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.832621:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada58a1c200. 02000000:00000001:2.0:1788572306.832624:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.832625:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.832627:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada516eea00. 00000100:00000001:2.0:1788572306.832629:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.832630:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.832635:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60411880 x1875449615962368/t219043334179(219043334179) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.832641:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.832643:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60411880 x1875449615962368/t219043334179(219043334179) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.832649:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.832650:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.832651:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada51753800. 02000000:00000001:2.0:1788572306.832653:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.832654:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.832655:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=333 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.832657:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.832671:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada58a1dc00. 02000000:00000001:2.0:1788572306.832674:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.832676:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.832677:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60411880. 00000100:00000001:2.0:1788572306.832680:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.832681:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.832686:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60413480 x1875449615962752/t219043334180(219043334180) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.832692:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.832695:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60413480 x1875449615962752/t219043334180(219043334180) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.832700:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.832701:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.832702:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45571400. 02000000:00000001:2.0:1788572306.832705:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.832706:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.832708:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=332 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.832709:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.832710:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada58a1d400. 02000000:00000001:2.0:1788572306.832712:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.832713:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.832714:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60413480. 00000100:00000001:2.0:1788572306.832717:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.832718:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.832722:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60412d80 x1875449615963136/t219043334181(219043334181) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.832735:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.832738:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60412d80 x1875449615963136/t219043334181(219043334181) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.832744:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.832745:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.832746:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45573800. 02000000:00000001:2.0:1788572306.832749:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.832749:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.832750:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=331 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.832752:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.832753:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada58a1d000. 02000000:00000001:2.0:1788572306.832755:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.832756:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.832757:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60412d80. 00000100:00000001:2.0:1788572306.832759:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.832760:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.832764:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60412300 x1875449615963520/t219043334182(219043334182) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.832770:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.832772:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60412300 x1875449615963520/t219043334182(219043334182) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.832777:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.832778:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.832780:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45570c00. 02000000:00000001:2.0:1788572306.832782:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.832783:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.832783:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=330 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.832785:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.832786:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada58a1c400. 02000000:00000001:2.0:1788572306.832788:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.832789:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.832791:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60412300. 00000100:00000001:2.0:1788572306.832793:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.832794:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:0.0:1788572306.836769:0:2382:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788572306.836777:0:2382:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788572306.836783:0:2382:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada60acaf20. 00000400:00000200:0.0:1788572306.836795:0:2382:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada5176a088 00000400:00000010:0.0:1788572306.836804:0:2382:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada5176a088. 00000100:00000001:0.0:1788572306.836810:0:2382:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:0.0:1788572306.836824:0:2382:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada60624700 x1875449616089856/t0(0) o9->lustre-OST0001-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572316 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788572306.836837:0:2382:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada60624700 x1875449616089856/t0(0) o9->lustre-OST0001-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572316 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572306.836847:0:2382:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788572306.836849:0:2382:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572306.836852:0:2382:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:0.0:1788572306.836866:0:2382:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788572306.836870:0:2382:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788572306.836872:0:2382:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572306.836875:0:2382:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788572306.836887:0:2382:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada48400c60 time=151 v=5 (5 5 5 5) 00000100:00001000:0.0:1788572306.836891:0:2382:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada48400c18 time=22 v=5 (1 0 0 0) 00000100:00000001:0.0:1788572306.836894:0:2382:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788572306.836896:0:2382:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788572306.836899:0:2382:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788572306.836903:0:2382:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572306.836907:0:2382:0:(client.c:3110:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788572306.836909:0:2382:0:(client.c:3122:ptlrpc_free_committed()) lustre-OST0001-osc-ffff9ada58b0a000: skip recheck: last_committed 4294967635 00000100:00000001:0.0:1788572306.836911:0:2382:0:(client.c:3123:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788572306.836913:0:2382:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572306.836918:0:2382:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada60624700 x1875449616089856/t0(0) o9->lustre-OST0001-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572316 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572306.836925:0:2382:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788572306.836928:0:2382:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788572306.836930:0:2382:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00080000:0.0:1788572306.836934:0:2382:0:(import.c:2057:ptlrpc_disconnect_idle_interpret()) @@@ inflight=1, refcount=3: rc = 0 req@ffff9ada60624700 x1875449616089856/t0(0) o9->lustre-OST0001-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572316 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572306.836943:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada48400800 lustre-OST0001_UUID: changing import state from CONNECTING to IDLE 00000100:00000040:0.0:1788572306.836948:0:2382:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada60624700 x1875449616089856/t0(0) o9->lustre-OST0001-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572316 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572306.836956:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616089856:192.168.202.113@tcp:9:kworker.0 00000100:00000001:0.0:1788572306.836961:0:2382:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572306.836965:0:2382:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788572306.836968:0:2382:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60624700 x1875449616089856/t0(0) o9->lustre-OST0001-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572316 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572306.836974:0:2382:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788572306.836977:0:2382:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788572306.836981:0:2382:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada456bba00. 02000000:00000001:0.0:1788572306.836986:0:2382:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788572306.836989:0:2382:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:0.0:1788572306.836991:0:2382:0:(genops.c:1223:class_import_put()) import ffff9ada48400800 refcount=2 obd=lustre-OST0001-osc-ffff9ada58b0a000 00000020:00000001:0.0:1788572306.836994:0:2382:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:0.0:1788572306.836997:0:2382:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada5379ef00. 02000000:00000001:0.0:1788572306.837003:0:2382:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788572306.837005:0:2382:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788572306.837007:0:2382:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60624700. 00000100:00000001:0.0:1788572306.837010:0:2382:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788572306.837012:0:2382:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572306.837014:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572306.837024:0:2382:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788572306.837025:0:2382:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572306.837028:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788572306.837030:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572306.837769:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60413b80 x1875449615963904/t219043334183(219043334183) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.837783:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.837786:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60413b80 x1875449615963904/t219043334183(219043334183) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.837792:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.837794:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.837804:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45570800. 02000000:00000001:2.0:1788572306.837809:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.837811:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.837812:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=329 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.837815:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.837816:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada48a61c00. 02000000:00000001:2.0:1788572306.837819:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.837820:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.837822:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60413b80. 00000100:00000001:2.0:1788572306.837825:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.837827:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.837832:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60410000 x1875449615964288/t219043334184(219043334184) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.837839:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.837842:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60410000 x1875449615964288/t219043334184(219043334184) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.837847:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.837848:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.837850:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45573000. 02000000:00000001:2.0:1788572306.837864:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.837865:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.837866:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=328 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.837868:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.837870:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada41481e00. 02000000:00000001:2.0:1788572306.837872:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.837874:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.837875:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60410000. 00000100:00000001:2.0:1788572306.837877:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.837878:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.837883:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60545500 x1875449615964672/t219043334185(219043334185) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.837890:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.837893:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60545500 x1875449615964672/t219043334185(219043334185) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.837900:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.837901:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.837902:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada42a2a000. 02000000:00000001:2.0:1788572306.837905:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.837906:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.837907:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=327 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.837908:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.837910:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada4551c400. 02000000:00000001:2.0:1788572306.837912:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.837912:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.837914:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60545500. 00000100:00000001:2.0:1788572306.837916:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.837917:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.837921:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60413800 x1875449615965056/t219043334186(219043334186) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.837927:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.837930:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60413800 x1875449615965056/t219043334186(219043334186) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.837936:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.837937:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.837938:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45572000. 02000000:00000001:2.0:1788572306.837940:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.837942:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.837943:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=326 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.837945:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.837946:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60ad1e00. 02000000:00000001:2.0:1788572306.837948:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.837949:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.837950:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60413800. 00000100:00000001:2.0:1788572306.837952:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.837953:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.837957:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60410a80 x1875449615965440/t219043334187(219043334187) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.837963:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.837966:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60410a80 x1875449615965440/t219043334187(219043334187) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.837971:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.837972:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.837974:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45573400. 02000000:00000001:2.0:1788572306.837975:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.837976:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.837977:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=325 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.837979:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.837980:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada58a1c600. 02000000:00000001:2.0:1788572306.837982:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.837983:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.837984:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60410a80. 00000100:00000001:2.0:1788572306.837986:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.837987:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.837992:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60411f80 x1875449615965824/t219043334188(219043334188) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.837997:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.838000:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60411f80 x1875449615965824/t219043334188(219043334188) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.838005:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.838007:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.838008:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45572c00. 02000000:00000001:2.0:1788572306.838011:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.838012:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.838013:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=324 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.838015:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.838016:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51632600. 02000000:00000001:2.0:1788572306.838020:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.838021:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.838022:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60411f80. 00000100:00000001:2.0:1788572306.838024:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.838025:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.838028:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60410e00 x1875449615966208/t219043334189(219043334189) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.838034:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.838036:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60410e00 x1875449615966208/t219043334189(219043334189) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.838052:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.838053:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.838054:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45571800. 02000000:00000001:2.0:1788572306.838056:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.838058:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.838059:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=323 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.838060:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.838062:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51632a00. 02000000:00000001:2.0:1788572306.838063:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.838065:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.838066:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60410e00. 00000100:00000001:2.0:1788572306.838068:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.838070:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.838074:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60412a00 x1875449615966592/t219043334190(219043334190) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.838081:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.838084:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60412a00 x1875449615966592/t219043334190(219043334190) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.838091:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.838093:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.838094:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45570000. 02000000:00000001:2.0:1788572306.838096:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.838097:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.838098:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=322 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.838100:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.838101:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51632000. 02000000:00000001:2.0:1788572306.838102:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.838103:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.838104:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60412a00. 00000100:00000001:2.0:1788572306.838106:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.838107:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.838111:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60545880 x1875449615966976/t219043334191(219043334191) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.838117:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.838120:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60545880 x1875449615966976/t219043334191(219043334191) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.838126:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.838127:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.838128:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada42a2b800. 02000000:00000001:2.0:1788572306.838131:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.838132:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.838133:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=321 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.838135:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.838136:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada4551c000. 02000000:00000001:2.0:1788572306.838138:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.838139:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.838140:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60545880. 00000100:00000001:2.0:1788572306.838141:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.838142:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.838147:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60412680 x1875449615967360/t219043334192(219043334192) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.838158:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.838161:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60412680 x1875449615967360/t219043334192(219043334192) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.838167:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.838168:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.838169:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45573c00. 02000000:00000001:2.0:1788572306.838171:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.838172:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.838173:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=320 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.838175:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.838176:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51633400. 02000000:00000001:2.0:1788572306.838179:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.838179:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.838181:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60412680. 00000100:00000001:2.0:1788572306.838183:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.838184:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.838188:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60410380 x1875449615967744/t219043334193(219043334193) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.838193:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.838196:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60410380 x1875449615967744/t219043334193(219043334193) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.838202:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.838203:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.838204:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45571c00. 02000000:00000001:2.0:1788572306.838206:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.838206:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.838207:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=319 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.838209:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.838210:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51633200. 02000000:00000001:2.0:1788572306.838212:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.838213:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.838214:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60410380. 00000100:00000001:2.0:1788572306.838216:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.838217:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.838222:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60411500 x1875449615968128/t219043334194(219043334194) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.839360:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.839370:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60411500 x1875449615968128/t219043334194(219043334194) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.839378:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.839381:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.839383:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45570400. 02000000:00000001:2.0:1788572306.839389:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.839391:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.839392:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=318 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.839395:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.839398:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51632400. 02000000:00000001:2.0:1788572306.839401:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.839403:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.839405:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60411500. 00000100:00000001:2.0:1788572306.839408:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.839410:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.839415:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45537b80 x1875449615968512/t219043334195(219043334195) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.839422:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.839425:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45537b80 x1875449615968512/t219043334195(219043334195) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.839432:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.839433:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.839435:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada454a6800. 02000000:00000001:2.0:1788572306.839437:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.839438:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.839439:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=317 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.839441:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.839444:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada5170e000. 02000000:00000001:2.0:1788572306.839446:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.839447:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.839449:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45537b80. 00000100:00000001:2.0:1788572306.839451:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.839452:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.839457:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45536300 x1875449615968896/t219043334196(219043334196) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.839463:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.839466:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45536300 x1875449615968896/t219043334196(219043334196) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.839472:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.839473:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.839475:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada454a6400. 02000000:00000001:2.0:1788572306.839477:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.839478:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.839480:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=316 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.839482:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.839483:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada5170ee00. 02000000:00000001:2.0:1788572306.839485:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.839486:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.839488:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45536300. 00000100:00000001:2.0:1788572306.839490:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.839491:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.839496:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45536d80 x1875449615969280/t219043334197(219043334197) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.839502:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.839505:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45536d80 x1875449615969280/t219043334197(219043334197) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.839512:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.839513:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.839514:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada454a5000. 02000000:00000001:2.0:1788572306.839517:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.839519:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.839520:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=315 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.839522:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.839523:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada5170e200. 02000000:00000001:2.0:1788572306.839525:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.839526:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.839527:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45536d80. 00000100:00000001:2.0:1788572306.839529:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.839531:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.839535:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45537480 x1875449615969664/t219043334198(219043334198) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.839542:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.839544:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45537480 x1875449615969664/t219043334198(219043334198) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.839550:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.839551:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.839553:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada454a6000. 02000000:00000001:2.0:1788572306.839555:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.839556:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.839557:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=314 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.839559:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.839561:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada5170f200. 02000000:00000001:2.0:1788572306.839563:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.839564:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.839565:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45537480. 00000100:00000001:2.0:1788572306.839567:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.839568:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.839573:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45536680 x1875449615970048/t219043334199(219043334199) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.839579:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.839581:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45536680 x1875449615970048/t219043334199(219043334199) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.839587:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.839588:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.839590:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada454a4c00. 02000000:00000001:2.0:1788572306.839592:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.839593:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.839594:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=313 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.839596:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.839597:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada5170f400. 02000000:00000001:2.0:1788572306.839599:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.839600:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.839602:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45536680. 00000100:00000001:2.0:1788572306.839604:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.839605:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.839610:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60544700 x1875449615970432/t219043334200(219043334200) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.839616:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.839619:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60544700 x1875449615970432/t219043334200(219043334200) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.839626:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.839627:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.839628:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada42a2b000. 02000000:00000001:2.0:1788572306.839632:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.839633:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.839634:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=312 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.839636:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.839638:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada4551d000. 02000000:00000001:2.0:1788572306.839640:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.839641:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.839642:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60544700. 00000100:00000001:2.0:1788572306.839645:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.839646:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.839651:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60413100 x1875449615970816/t219043334201(219043334201) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.839656:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.840179:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60413100 x1875449615970816/t219043334201(219043334201) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.840189:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.840191:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.840193:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45572800. 02000000:00000001:2.0:1788572306.840196:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.840198:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.840199:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=311 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.840201:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.840203:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51632200. 02000000:00000001:2.0:1788572306.840206:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.840207:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.840209:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60413100. 00000100:00000001:2.0:1788572306.840212:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.840213:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.840218:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60410700 x1875449615971200/t219043334202(219043334202) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.840224:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.840300:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60410700 x1875449615971200/t219043334202(219043334202) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.840307:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.840308:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.840310:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45572400. 02000000:00000001:2.0:1788572306.840312:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.840314:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.840315:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=310 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.840317:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.840318:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60ad1000. 02000000:00000001:2.0:1788572306.840320:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.840321:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.840322:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60410700. 00000100:00000001:2.0:1788572306.840324:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.840325:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.840330:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60411180 x1875449615971584/t219043334203(219043334203) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.840340:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.840343:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60411180 x1875449615971584/t219043334203(219043334203) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.840348:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.840349:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.840350:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45571000. 02000000:00000001:2.0:1788572306.840352:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.840353:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.840354:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=309 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.840356:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.840357:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada5340cc00. 02000000:00000001:2.0:1788572306.840361:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.840362:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.840363:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60411180. 00000100:00000001:2.0:1788572306.840365:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.840366:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.840371:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60411c00 x1875449615971968/t219043334204(219043334204) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.840377:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.840379:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60411c00 x1875449615971968/t219043334204(219043334204) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.840385:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.840386:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.840387:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45439c00. 02000000:00000001:2.0:1788572306.840408:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.840410:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.840411:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=308 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.840413:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.840414:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60a24a00. 02000000:00000001:2.0:1788572306.840417:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.840418:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.840419:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60411c00. 00000100:00000001:2.0:1788572306.840422:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.840423:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.840428:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada607b0a80 x1875449615972352/t219043334205(219043334205) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.840434:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.840436:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada607b0a80 x1875449615972352/t219043334205(219043334205) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.840441:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.840443:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.840444:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45438400. 02000000:00000001:2.0:1788572306.840446:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.840447:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.840448:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=307 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.840449:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.840450:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60a24c00. 02000000:00000001:2.0:1788572306.840452:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.840453:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.840454:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada607b0a80. 00000100:00000001:2.0:1788572306.840456:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.840457:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.840461:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada607b1f80 x1875449615972736/t219043334206(219043334206) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.840466:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.840469:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada607b1f80 x1875449615972736/t219043334206(219043334206) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.840474:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.840475:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.840476:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4543a800. 02000000:00000001:2.0:1788572306.840479:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.840480:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.840481:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=306 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.840482:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.840484:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60a25800. 02000000:00000001:2.0:1788572306.840486:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.840487:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.840488:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada607b1f80. 00000100:00000001:2.0:1788572306.840490:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.840491:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.840495:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada607b0e00 x1875449615973120/t219043334207(219043334207) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.840501:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.840503:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada607b0e00 x1875449615973120/t219043334207(219043334207) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.840509:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.840510:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.840511:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4543a400. 02000000:00000001:2.0:1788572306.840513:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.840514:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.840516:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=305 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.840517:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.840519:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60a25a00. 02000000:00000001:2.0:1788572306.840521:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.840521:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.840523:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada607b0e00. 00000100:00000001:2.0:1788572306.840525:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.840526:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.840531:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada607b2a00 x1875449615973504/t219043334208(219043334208) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.840537:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.840539:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada607b2a00 x1875449615973504/t219043334208(219043334208) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.840545:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.840547:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.840548:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45439000. 02000000:00000001:2.0:1788572306.840550:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.840552:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.840553:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=304 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.840554:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.840556:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60a25c00. 02000000:00000001:2.0:1788572306.840557:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.840558:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.840559:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada607b2a00. 00000100:00000001:2.0:1788572306.840562:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.840563:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.840567:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada607b2680 x1875449615973888/t219043334209(219043334209) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.840573:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.840575:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada607b2680 x1875449615973888/t219043334209(219043334209) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.840581:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.840582:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.840584:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45439400. 02000000:00000001:2.0:1788572306.840585:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.840586:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.840587:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=303 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.840589:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.840591:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60a24400. 02000000:00000001:2.0:1788572306.840592:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.840593:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.840595:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada607b2680. 00000100:00000001:2.0:1788572306.840597:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.840598:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.840603:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada607b0380 x1875449615974272/t219043334210(219043334210) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.840609:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.840612:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada607b0380 x1875449615974272/t219043334210(219043334210) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.840618:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.840619:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.840621:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4543b800. 02000000:00000001:2.0:1788572306.840624:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.840625:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.840627:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=302 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.840628:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.840630:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60a24200. 02000000:00000001:2.0:1788572306.840632:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.840633:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.840634:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada607b0380. 00000100:00000001:2.0:1788572306.840637:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.840638:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.840643:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada607b1500 x1875449615974656/t219043334211(219043334211) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.840649:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.840651:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada607b1500 x1875449615974656/t219043334211(219043334211) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842124:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.842132:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.842136:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45438c00. 02000000:00000001:2.0:1788572306.842145:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.842148:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.842149:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=301 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.842152:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.842155:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60a25400. 02000000:00000001:2.0:1788572306.842158:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.842160:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.842162:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada607b1500. 00000100:00000001:2.0:1788572306.842165:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.842167:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.842179:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada607b3100 x1875449615975040/t219043334212(219043334212) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842189:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.842192:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada607b3100 x1875449615975040/t219043334212(219043334212) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842199:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.842200:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.842202:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45438800. 02000000:00000001:2.0:1788572306.842204:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.842205:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.842206:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=300 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.842208:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.842209:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60a25e00. 02000000:00000001:2.0:1788572306.842211:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.842212:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.842213:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada607b3100. 00000100:00000001:2.0:1788572306.842216:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.842216:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.842221:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada607b0700 x1875449615975424/t219043334213(219043334213) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842227:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.842230:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada607b0700 x1875449615975424/t219043334213(219043334213) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842235:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.842236:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.842237:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4543b000. 02000000:00000001:2.0:1788572306.842239:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.842240:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.842242:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=299 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.842244:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.842245:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60a25000. 02000000:00000001:2.0:1788572306.842247:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.842248:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.842249:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada607b0700. 00000100:00000001:2.0:1788572306.842251:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.842252:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.842256:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada607b1180 x1875449615975808/t219043334214(219043334214) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842267:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.842269:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada607b1180 x1875449615975808/t219043334214(219043334214) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842275:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.842276:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.842278:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4543a000. 02000000:00000001:2.0:1788572306.842280:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.842281:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.842282:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=298 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.842284:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.842285:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60a24600. 02000000:00000001:2.0:1788572306.842287:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.842288:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.842289:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada607b1180. 00000100:00000001:2.0:1788572306.842291:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.842292:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.842297:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45535180 x1875449615976192/t219043334215(219043334215) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842303:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.842305:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45535180 x1875449615976192/t219043334215(219043334215) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842311:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.842312:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.842313:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada454a7800. 02000000:00000001:2.0:1788572306.842331:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.842332:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.842333:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=297 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.842335:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.842337:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada5170e600. 02000000:00000001:2.0:1788572306.842339:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.842340:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.842342:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45535180. 00000100:00000001:2.0:1788572306.842345:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.842346:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.842351:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45535880 x1875449615976576/t219043334216(219043334216) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842357:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.842359:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45535880 x1875449615976576/t219043334216(219043334216) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842363:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.842365:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.842366:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4558b800. 02000000:00000001:2.0:1788572306.842369:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.842369:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.842370:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=296 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.842372:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.842373:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada5068a800. 02000000:00000001:2.0:1788572306.842375:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.842376:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.842377:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45535880. 00000100:00000001:2.0:1788572306.842379:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.842380:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.842384:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45534700 x1875449615976960/t219043334217(219043334217) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842390:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.842392:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45534700 x1875449615976960/t219043334217(219043334217) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842398:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.842399:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.842400:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45589400. 02000000:00000001:2.0:1788572306.842403:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.842404:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.842405:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=295 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.842406:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.842409:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada516ea000. 02000000:00000001:2.0:1788572306.842412:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.842413:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.842414:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45534700. 00000100:00000001:2.0:1788572306.842417:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.842418:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.842422:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4557ce00 x1875449615977344/t219043334218(219043334218) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842429:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.842431:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4557ce00 x1875449615977344/t219043334218(219043334218) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842437:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.842438:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.842440:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45589000. 02000000:00000001:2.0:1788572306.842442:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.842443:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.842444:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=294 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.842447:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.842448:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada516ebe00. 02000000:00000001:2.0:1788572306.842450:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.842451:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.842453:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4557ce00. 00000100:00000001:2.0:1788572306.842456:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.842457:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.842461:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4557ea00 x1875449615977728/t219043334219(219043334219) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842467:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.842469:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4557ea00 x1875449615977728/t219043334219(219043334219) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842474:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.842475:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.842477:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4558a400. 02000000:00000001:2.0:1788572306.842479:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.842481:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.842482:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=293 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.842483:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.842485:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada516ea400. 02000000:00000001:2.0:1788572306.842486:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.842487:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.842488:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4557ea00. 00000100:00000001:2.0:1788572306.842491:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.842492:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.842497:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4557e680 x1875449615978112/t219043334220(219043334220) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842502:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.842505:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4557e680 x1875449615978112/t219043334220(219043334220) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842511:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.842512:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.842513:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4558a800. 02000000:00000001:2.0:1788572306.842515:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.842516:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.842517:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=292 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.842520:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.842521:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada516ea600. 02000000:00000001:2.0:1788572306.842523:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.842524:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.842525:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4557e680. 00000100:00000001:2.0:1788572306.842527:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.842528:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.842533:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4557c380 x1875449615978496/t219043334221(219043334221) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842539:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.842542:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4557c380 x1875449615978496/t219043334221(219043334221) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842548:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.842550:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.842551:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45588400. 02000000:00000001:2.0:1788572306.842554:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.842555:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.842556:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=291 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.842558:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.842559:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada516ea800. 02000000:00000001:2.0:1788572306.842561:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.842562:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.842564:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4557c380. 00000100:00000001:2.0:1788572306.842566:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.842567:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.842571:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4557d500 x1875449615978880/t219043334222(219043334222) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842578:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.842580:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4557d500 x1875449615978880/t219043334222(219043334222) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842586:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.842587:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.842589:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45589c00. 02000000:00000001:2.0:1788572306.842591:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.842592:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.842593:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=290 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.842595:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.842596:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada516eb200. 02000000:00000001:2.0:1788572306.842598:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.842599:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.842600:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4557d500. 00000100:00000001:2.0:1788572306.842602:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.842603:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.842607:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4557f100 x1875449615979264/t219043334223(219043334223) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842614:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.842617:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4557f100 x1875449615979264/t219043334223(219043334223) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842623:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.842625:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.842626:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4558bc00. 02000000:00000001:2.0:1788572306.842629:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.842630:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.842631:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=289 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.842633:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.842635:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada516eac00. 02000000:00000001:2.0:1788572306.842637:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.842638:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.842640:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4557f100. 00000100:00000001:2.0:1788572306.842642:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.842643:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.842648:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada455a4a80 x1875449615979648/t219043334224(219043334224) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842655:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.842659:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455a4a80 x1875449615979648/t219043334224(219043334224) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842665:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.842666:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.842668:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada512e3000. 02000000:00000001:2.0:1788572306.842671:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.842672:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.842674:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=288 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.842676:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.842677:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada519ffa00. 02000000:00000001:2.0:1788572306.842679:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.842681:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.842682:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455a4a80. 00000100:00000001:2.0:1788572306.842684:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.842685:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.842690:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4557c700 x1875449615980032/t219043334225(219043334225) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842702:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.842705:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4557c700 x1875449615980032/t219043334225(219043334225) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842711:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.842712:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.842713:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45588000. 02000000:00000001:2.0:1788572306.842715:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.842716:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.842717:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=287 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.842718:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.842719:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada516eb800. 02000000:00000001:2.0:1788572306.842721:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.842722:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.842724:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4557c700. 00000100:00000001:2.0:1788572306.842726:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.842727:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.842731:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4557d180 x1875449615980416/t219043334226(219043334226) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842737:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.842739:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4557d180 x1875449615980416/t219043334226(219043334226) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842744:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.842745:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.842747:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45589800. 02000000:00000001:2.0:1788572306.842748:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.842749:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.842750:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=286 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.842752:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.842753:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada5170f600. 02000000:00000001:2.0:1788572306.842755:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.842756:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.842758:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4557d180. 00000100:00000001:2.0:1788572306.842759:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.842760:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.842765:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada607b1880 x1875449615980800/t219043334227(219043334227) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842771:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.842773:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada607b1880 x1875449615980800/t219043334227(219043334227) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842779:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.842780:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.842781:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4543b400. 02000000:00000001:2.0:1788572306.842784:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.842785:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.842786:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=285 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.842788:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.842790:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60a24e00. 02000000:00000001:2.0:1788572306.842791:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.842792:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.842794:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada607b1880. 00000100:00000001:2.0:1788572306.842796:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.842797:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.842802:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada607b3480 x1875449615981184/t219043334228(219043334228) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842809:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.842811:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada607b3480 x1875449615981184/t219043334228(219043334228) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842816:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.842818:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.842819:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4543ac00. 02000000:00000001:2.0:1788572306.842822:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.842823:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.842824:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=284 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.842826:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.842828:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51632c00. 02000000:00000001:2.0:1788572306.842831:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.842832:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.842833:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada607b3480. 00000100:00000001:2.0:1788572306.842835:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.842836:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.842841:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada607b2d80 x1875449615981568/t219043334229(219043334229) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842847:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.842850:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada607b2d80 x1875449615981568/t219043334229(219043334229) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842883:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.842885:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.842886:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45439800. 02000000:00000001:2.0:1788572306.842889:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.842890:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.842891:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=283 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.842894:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.842895:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada48746400. 02000000:00000001:2.0:1788572306.842901:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.842902:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.842903:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada607b2d80. 00000100:00000001:2.0:1788572306.842906:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.842907:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.842912:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada607b2300 x1875449615981952/t219043334230(219043334230) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842917:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.842920:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada607b2300 x1875449615981952/t219043334230(219043334230) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842925:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.842926:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.842928:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45438000. 02000000:00000001:2.0:1788572306.842930:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.842931:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.842932:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=282 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.842934:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.842935:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada48747c00. 02000000:00000001:2.0:1788572306.842937:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.842939:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.842940:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada607b2300. 00000100:00000001:2.0:1788572306.842942:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.842943:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.842947:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada607b3b80 x1875449615982336/t219043334231(219043334231) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842953:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.842955:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada607b3b80 x1875449615982336/t219043334231(219043334231) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842961:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.842962:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.842963:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4543bc00. 02000000:00000001:2.0:1788572306.842965:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.842966:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.842967:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=281 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.842969:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.842970:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60a25200. 02000000:00000001:2.0:1788572306.842972:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.842973:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.842974:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada607b3b80. 00000100:00000001:2.0:1788572306.842976:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.842978:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.842983:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada607b1c00 x1875449615982720/t219043334232(219043334232) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842989:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.842991:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada607b1c00 x1875449615982720/t219043334232(219043334232) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.842997:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.842999:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.843000:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada50b6b800. 02000000:00000001:2.0:1788572306.843003:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.843004:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.843005:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=280 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.843007:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.843008:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada603e8600. 02000000:00000001:2.0:1788572306.843010:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.843011:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.843013:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada607b1c00. 00000100:00000001:2.0:1788572306.843015:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.843016:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.843021:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60545180 x1875449615983104/t219043334233(219043334233) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.843028:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.843031:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60545180 x1875449615983104/t219043334233(219043334233) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.843036:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.843049:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.843050:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada42a29c00. 02000000:00000001:2.0:1788572306.843054:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.843055:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.843056:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=279 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.843058:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.843059:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada4551ce00. 02000000:00000001:2.0:1788572306.843061:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.843062:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.843063:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60545180. 00000100:00000001:2.0:1788572306.843065:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.843066:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.843071:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60547480 x1875449615983488/t219043334234(219043334234) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.843077:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.843080:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60547480 x1875449615983488/t219043334234(219043334234) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.843085:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.843086:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.843088:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada42a2ac00. 02000000:00000001:2.0:1788572306.843090:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.843092:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.843093:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=278 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.843095:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.843096:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada4551dc00. 02000000:00000001:2.0:1788572306.843098:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.843100:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.843101:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60547480. 00000100:00000001:2.0:1788572306.843103:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.843104:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.843109:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60546d80 x1875449615983872/t219043334235(219043334235) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.843115:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.843118:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60546d80 x1875449615983872/t219043334235(219043334235) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.843124:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.843125:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.843126:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada42a2bc00. 02000000:00000001:2.0:1788572306.843129:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.843131:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.843132:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=277 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.843134:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.843135:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada48747a00. 02000000:00000001:2.0:1788572306.843137:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.843138:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.843140:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60546d80. 00000100:00000001:2.0:1788572306.843142:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.843144:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.843149:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60547100 x1875449615984256/t219043334236(219043334236) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.843160:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.843163:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60547100 x1875449615984256/t219043334236(219043334236) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.843169:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.843170:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.843172:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada42a2a400. 02000000:00000001:2.0:1788572306.843174:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.843175:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.843177:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=276 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.843179:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.843180:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada53333e00. 02000000:00000001:2.0:1788572306.843183:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.843184:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.843186:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60547100. 00000100:00000001:2.0:1788572306.843188:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.843188:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.843193:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60547800 x1875449615984640/t219043334237(219043334237) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.843200:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.843203:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60547800 x1875449615984640/t219043334237(219043334237) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.843207:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.843209:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.843210:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada42a29400. 02000000:00000001:2.0:1788572306.843212:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.843213:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.843215:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=275 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.843216:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.843218:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada517d4000. 02000000:00000001:2.0:1788572306.843221:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.843222:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.843224:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60547800. 00000100:00000001:2.0:1788572306.843226:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.843227:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.843232:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60546680 x1875449615985024/t219043334238(219043334238) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.843237:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.843240:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60546680 x1875449615985024/t219043334238(219043334238) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572314 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.843246:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.843247:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.843249:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada42a2b400. 02000000:00000001:2.0:1788572306.843251:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.843252:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.843253:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=274 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.843255:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.843256:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada517d4400. 02000000:00000001:2.0:1788572306.843258:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.843259:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.843260:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60546680. 00000100:00000001:2.0:1788572306.843262:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.843263:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.843267:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60546300 x1875449615985408/t219043334239(219043334239) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.843273:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.843276:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60546300 x1875449615985408/t219043334239(219043334239) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.843281:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.843282:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.843284:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada42a28000. 02000000:00000001:2.0:1788572306.843286:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.843287:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.843288:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=273 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.843290:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.843291:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada517d5400. 02000000:00000001:2.0:1788572306.843293:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.843294:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.843295:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60546300. 00000100:00000001:2.0:1788572306.843297:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.843298:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.843302:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60544380 x1875449615985792/t219043334240(219043334240) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.843308:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.843311:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60544380 x1875449615985792/t219043334240(219043334240) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.843316:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.843318:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.843319:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada42a28c00. 02000000:00000001:2.0:1788572306.843321:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000800:00000001:3.0:1788572306.844381:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572306.844400:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572306.844405:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572306.844408:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572306.844418:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572306.844426:0:2378:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572306.844439:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572306.844445:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572306.844451:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572306.844456:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 25 MB=0x6a9b624294100 00000400:00000200:3.0:1788572306.844462:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572306.844469:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.202.113@tcp of length 224/224 into md 0x35685 [1] + 224 00000800:00000001:3.0:1788572306.844477:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572306.844482:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572306.844487:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572306.844492:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572306.844502:0:2378:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572306.844512:0:2378:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada60626680 x1875449616089344/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572322 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572306.844523:0:2378:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada60626680 x1875449616089344/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572322 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572306.844533:0:2378:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572306.844542:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572306.844546:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572306.844547:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572306.844549:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572306.844552:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572306.844556:0:2378:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572306.844560:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572306.844562:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572306.844565:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572306.844568:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 10 MB=0x6a9b624294200 00000400:00000200:3.0:1788572306.844572:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572306.844576:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.113@tcp of length 224/224 into md 0x35695 [1] + 224 00000800:00000001:3.0:1788572306.844580:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572306.844586:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572306.844589:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572306.844592:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572306.844595:0:2378:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572306.844601:0:2378:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada60626a00 x1875449616089600/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572322 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572306.844608:0:2378:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada60626a00 x1875449616089600/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572322 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572306.844614:0:2378:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572306.844618:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572306.844620:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:2.0:1788572306.844796:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.845924:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=272 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.845932:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.845938:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada517d4600. 02000000:00000001:2.0:1788572306.845946:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.845948:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.845951:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60544380. 00000100:00000001:2.0:1788572306.845955:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.845957:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.845970:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada455a6a00 x1875449615986176/t219043334241(219043334241) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.845979:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.845983:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455a6a00 x1875449615986176/t219043334241(219043334241) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.845990:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.845992:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.845994:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada512e0000. 02000000:00000001:2.0:1788572306.845997:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.846000:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.846001:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=271 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.846003:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.846004:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada519fec00. 02000000:00000001:2.0:1788572306.846007:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.846008:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.846009:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455a6a00. 00000100:00000001:2.0:1788572306.846012:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.846013:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.846018:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4557d880 x1875449615986560/t219043334242(219043334242) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.846025:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.846028:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4557d880 x1875449615986560/t219043334242(219043334242) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.846034:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.846035:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.846037:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4558ac00. 02000000:00000001:2.0:1788572306.850824:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.850831:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.850833:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=270 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.850842:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.850847:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada4283d400. 02000000:00000001:2.0:1788572306.850867:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.850870:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.850873:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4557d880. 00000100:00000001:2.0:1788572306.850880:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.850882:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.850900:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4557f480 x1875449615986944/t219043334243(219043334243) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.850911:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.850915:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4557f480 x1875449615986944/t219043334243(219043334243) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.850923:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.850927:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.850929:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4558b400. 02000000:00000001:2.0:1788572306.850932:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.850934:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.850935:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=269 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.850937:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.850939:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada516eaa00. 02000000:00000001:2.0:1788572306.850942:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.850943:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.850945:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4557f480. 00000100:00000001:2.0:1788572306.850948:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.850949:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.850955:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada607b0000 x1875449615987328/t219043334244(219043334244) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.850962:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.850965:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada607b0000 x1875449615987328/t219043334244(219043334244) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.850971:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.850973:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.850974:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada50b69000. 02000000:00000001:2.0:1788572306.850977:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.850978:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.850980:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=268 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.850981:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.850983:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada603e9200. 02000000:00000001:2.0:1788572306.850985:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.850986:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.850988:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada607b0000. 00000100:00000001:2.0:1788572306.850990:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.850991:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.850996:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada607b3800 x1875449615987712/t219043334245(219043334245) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851003:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.851005:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada607b3800 x1875449615987712/t219043334245(219043334245) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851013:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.851014:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.851016:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada50b68400. 02000000:00000001:2.0:1788572306.851018:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.851019:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.851020:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=267 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.851022:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.851023:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada603e9e00. 02000000:00000001:2.0:1788572306.851025:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.851027:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.851028:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada607b3800. 00000100:00000001:2.0:1788572306.851031:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.851032:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.851036:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60774a80 x1875449615988096/t219043334246(219043334246) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851054:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.851057:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60774a80 x1875449615988096/t219043334246(219043334246) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851064:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.851065:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.851067:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada50b69800. 02000000:00000001:2.0:1788572306.851069:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.851070:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.851072:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=266 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.851074:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.851075:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada603e8c00. 02000000:00000001:2.0:1788572306.851077:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.851078:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.851080:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60774a80. 00000100:00000001:2.0:1788572306.851082:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.851083:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.851088:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60775f80 x1875449615988480/t219043334247(219043334247) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851100:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.851103:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60775f80 x1875449615988480/t219043334247(219043334247) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851109:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.851110:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.851112:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada50b68800. 02000000:00000001:2.0:1788572306.851114:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.851115:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.851116:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=265 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.851118:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.851119:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada603e8400. 02000000:00000001:2.0:1788572306.851121:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.851122:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.851124:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60775f80. 00000100:00000001:2.0:1788572306.851127:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.851128:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.851133:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4557ed80 x1875449615988864/t219043334248(219043334248) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851140:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.851143:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4557ed80 x1875449615988864/t219043334248(219043334248) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851149:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.851151:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.851152:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4558a000. 02000000:00000001:2.0:1788572306.851154:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.851155:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.851156:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=264 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.851158:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.851160:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada600aea00. 02000000:00000001:2.0:1788572306.851164:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.851165:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.851167:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4557ed80. 00000100:00000001:2.0:1788572306.851170:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.851171:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.851176:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4557e300 x1875449615989248/t219043334249(219043334249) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851183:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.851186:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4557e300 x1875449615989248/t219043334249(219043334249) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851193:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.851194:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.851196:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4558b000. 02000000:00000001:2.0:1788572306.851198:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.851199:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.851200:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=263 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.851202:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.851204:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada600ae800. 02000000:00000001:2.0:1788572306.851205:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.851207:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.851208:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4557e300. 00000100:00000001:2.0:1788572306.851210:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.851211:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.851216:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4557fb80 x1875449615989632/t219043334250(219043334250) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851222:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.851225:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4557fb80 x1875449615989632/t219043334250(219043334250) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851243:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.851245:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.851246:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45588800. 02000000:00000001:2.0:1788572306.851249:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.851250:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.851251:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=262 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.851253:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.851255:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada600af800. 02000000:00000001:2.0:1788572306.851259:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.851260:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.851261:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4557fb80. 00000100:00000001:2.0:1788572306.851263:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.851264:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.851269:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4557dc00 x1875449615990016/t219043334251(219043334251) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851276:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.851278:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4557dc00 x1875449615990016/t219043334251(219043334251) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851285:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.851286:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.851287:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45588c00. 02000000:00000001:2.0:1788572306.851290:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.851291:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.851292:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=261 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.851294:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.851295:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada600ae600. 02000000:00000001:2.0:1788572306.851297:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.851298:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.851300:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4557dc00. 00000100:00000001:2.0:1788572306.851302:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.851303:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.851308:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4557c000 x1875449615990400/t219043334252(219043334252) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851315:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.851318:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4557c000 x1875449615990400/t219043334252(219043334252) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851324:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.851325:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.851326:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada50609400. 02000000:00000001:2.0:1788572306.851329:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.851331:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.851332:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=260 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.851334:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.851335:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada4283ca00. 02000000:00000001:2.0:1788572306.851338:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.851339:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.851340:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4557c000. 00000100:00000001:2.0:1788572306.851342:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.851344:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.851349:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4557f800 x1875449615990784/t219043334253(219043334253) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851354:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.851357:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4557f800 x1875449615990784/t219043334253(219043334253) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851363:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.851364:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.851365:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada5060ac00. 02000000:00000001:2.0:1788572306.851368:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.851369:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.851370:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=259 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.851372:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.851374:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51656600. 02000000:00000001:2.0:1788572306.851376:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.851377:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.851378:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4557f800. 00000100:00000001:2.0:1788572306.851380:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.851381:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.851386:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4557ca80 x1875449615991168/t219043334254(219043334254) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851392:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.851395:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4557ca80 x1875449615991168/t219043334254(219043334254) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851401:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.851403:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.851404:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada50608800. 02000000:00000001:2.0:1788572306.851406:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.851407:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.851408:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=258 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.851410:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.851412:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51657600. 02000000:00000001:2.0:1788572306.851414:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.851415:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.851416:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4557ca80. 00000100:00000001:2.0:1788572306.851418:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.851420:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.851424:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4557df80 x1875449615991552/t219043334255(219043334255) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851431:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.851433:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4557df80 x1875449615991552/t219043334255(219043334255) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851440:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.851441:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.851442:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada50608000. 02000000:00000001:2.0:1788572306.851444:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.851445:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.851446:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=257 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.851448:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.851449:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51657000. 02000000:00000001:2.0:1788572306.851451:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.851451:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.851453:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4557df80. 00000100:00000001:2.0:1788572306.851455:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.851456:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.851460:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60743800 x1875449615991936/t219043334256(219043334256) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851466:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.851468:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60743800 x1875449615991936/t219043334256(219043334256) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851474:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.851476:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.851477:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada5060b400. 02000000:00000001:2.0:1788572306.851480:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.851481:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.851483:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=256 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.851484:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.851486:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51657800. 02000000:00000001:2.0:1788572306.851488:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.851489:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.851490:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60743800. 00000100:00000001:2.0:1788572306.851493:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.851494:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.851499:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60740a80 x1875449615992320/t219043334257(219043334257) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851505:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.851508:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60740a80 x1875449615992320/t219043334257(219043334257) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851514:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.851515:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.851516:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada50608c00. 02000000:00000001:2.0:1788572306.851519:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.851519:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.851521:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=255 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.851523:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.851524:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51657a00. 02000000:00000001:2.0:1788572306.851526:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.851527:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.851528:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60740a80. 00000100:00000001:2.0:1788572306.851531:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.851532:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.851536:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60741f80 x1875449615992704/t219043334258(219043334258) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851547:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.851551:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60741f80 x1875449615992704/t219043334258(219043334258) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851557:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.851558:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.851560:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada50609800. 02000000:00000001:2.0:1788572306.851562:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.851563:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.851564:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=254 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.851566:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.851567:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51657400. 02000000:00000001:2.0:1788572306.851569:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.851570:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.851571:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60741f80. 00000100:00000001:2.0:1788572306.851573:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.851575:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.851580:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60740e00 x1875449615993088/t219043334259(219043334259) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851586:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.851589:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60740e00 x1875449615993088/t219043334259(219043334259) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851594:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.851595:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.851597:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada5060bc00. 02000000:00000001:2.0:1788572306.851599:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.851600:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.851601:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=253 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.851603:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.851604:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51657e00. 02000000:00000001:2.0:1788572306.851606:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.851607:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.851609:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60740e00. 00000100:00000001:2.0:1788572306.851611:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.851612:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.851616:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60742a00 x1875449615993472/t219043334260(219043334260) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851623:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.851625:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60742a00 x1875449615993472/t219043334260(219043334260) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851632:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.851633:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.851634:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada5060b000. 02000000:00000001:2.0:1788572306.851637:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.851638:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.851639:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=252 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.851641:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.851643:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51656200. 02000000:00000001:2.0:1788572306.851644:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.851645:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.851647:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60742a00. 00000100:00000001:2.0:1788572306.851649:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.851650:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.851655:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60742680 x1875449615993856/t219043334261(219043334261) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851675:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.851679:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60742680 x1875449615993856/t219043334261(219043334261) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851685:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.851686:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.851688:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4653d400. 02000000:00000001:2.0:1788572306.851692:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.851692:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.851694:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=251 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.851695:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.851697:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada600afa00. 02000000:00000001:2.0:1788572306.851700:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.851701:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.851702:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60742680. 00000100:00000001:2.0:1788572306.851704:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.851705:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.851710:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60740380 x1875449615994240/t219043334262(219043334262) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851718:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.851721:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60740380 x1875449615994240/t219043334262(219043334262) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851727:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.851729:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.851730:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4653ec00. 02000000:00000001:2.0:1788572306.851733:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.851734:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.851736:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=250 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.851738:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.851739:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada48f89000. 02000000:00000001:2.0:1788572306.851741:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.851742:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.851743:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60740380. 00000100:00000001:2.0:1788572306.851746:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.851747:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.851752:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60741500 x1875449615994624/t219043334263(219043334263) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851759:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.851761:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60741500 x1875449615994624/t219043334263(219043334263) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851767:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.851768:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.851769:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4653e400. 02000000:00000001:2.0:1788572306.851771:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.851774:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.851774:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=249 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.851776:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.851777:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada5329aa00. 02000000:00000001:2.0:1788572306.851781:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.851782:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.851783:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60741500. 00000100:00000001:2.0:1788572306.851786:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.851787:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.851793:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60743100 x1875449615995008/t219043334264(219043334264) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851799:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.851802:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60743100 x1875449615995008/t219043334264(219043334264) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851808:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.851809:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.851811:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4653e000. 02000000:00000001:2.0:1788572306.851813:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.851815:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.851815:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=248 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.851817:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.851819:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada5329a400. 02000000:00000001:2.0:1788572306.851821:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.851822:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.851823:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60743100. 00000100:00000001:2.0:1788572306.851826:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.851827:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.851832:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60740700 x1875449615995392/t219043334265(219043334265) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851837:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.851840:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60740700 x1875449615995392/t219043334265(219043334265) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851846:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.851847:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.851849:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4653dc00. 02000000:00000001:2.0:1788572306.851851:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.851888:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.851890:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=247 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.851892:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.851894:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51657200. 02000000:00000001:2.0:1788572306.851896:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.851897:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.851899:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60740700. 00000100:00000001:2.0:1788572306.851901:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.851902:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.851907:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60741180 x1875449615995776/t219043334266(219043334266) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851914:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.851917:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60741180 x1875449615995776/t219043334266(219043334266) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851924:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.851926:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.851927:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4653e800. 02000000:00000001:2.0:1788572306.851930:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.851931:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.851932:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=246 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.851934:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.851935:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada501d1800. 02000000:00000001:2.0:1788572306.851939:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.851941:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.851942:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60741180. 00000100:00000001:2.0:1788572306.851944:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.851945:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.851950:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60741880 x1875449615996160/t219043334267(219043334267) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851957:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.851961:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60741880 x1875449615996160/t219043334267(219043334267) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.851968:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.851969:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.851971:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4653f400. 02000000:00000001:2.0:1788572306.851975:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.851976:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.851978:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=245 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.851979:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.851981:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada5123fc00. 02000000:00000001:2.0:1788572306.851983:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.851985:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.851986:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60741880. 00000100:00000001:2.0:1788572306.851988:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.851990:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.851995:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60743480 x1875449615996544/t219043334268(219043334268) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852001:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.852004:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60743480 x1875449615996544/t219043334268(219043334268) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852010:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.852012:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.852013:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4653f000. 02000000:00000001:2.0:1788572306.852016:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.852017:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.852018:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=244 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.852020:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.852021:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada5123f200. 02000000:00000001:2.0:1788572306.852023:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.852024:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.852025:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60743480. 00000100:00000001:2.0:1788572306.852028:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.852029:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.852034:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60742d80 x1875449615996928/t219043334269(219043334269) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852059:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.852063:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60742d80 x1875449615996928/t219043334269(219043334269) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852069:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.852070:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.852072:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4653d800. 02000000:00000001:2.0:1788572306.852075:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.852076:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.852077:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=243 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.852079:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.852080:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada5329a600. 02000000:00000001:2.0:1788572306.852082:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.852083:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.852085:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60742d80. 00000100:00000001:2.0:1788572306.852087:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.852088:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.852093:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60742300 x1875449615997312/t219043334270(219043334270) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852098:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.852101:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60742300 x1875449615997312/t219043334270(219043334270) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852108:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.852109:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.852110:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4653c800. 02000000:00000001:2.0:1788572306.852113:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.852114:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.852115:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=242 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.852117:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.852118:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51920400. 02000000:00000001:2.0:1788572306.852121:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.852122:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.852124:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60742300. 00000100:00000001:2.0:1788572306.852126:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.852127:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.852132:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60743b80 x1875449615997696/t219043334271(219043334271) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852138:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.852141:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60743b80 x1875449615997696/t219043334271(219043334271) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852147:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.852148:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.852149:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4653cc00. 02000000:00000001:2.0:1788572306.852152:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.852153:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.852154:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=241 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.852156:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.852157:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51921200. 02000000:00000001:2.0:1788572306.852159:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.852160:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.852162:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60743b80. 00000100:00000001:2.0:1788572306.852164:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.852165:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.852170:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60774e00 x1875449615998080/t219043334272(219043334272) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852177:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.852180:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60774e00 x1875449615998080/t219043334272(219043334272) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852186:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.852187:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.852188:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada50b6b000. 02000000:00000001:2.0:1788572306.852191:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.852192:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.852193:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=240 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.852195:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.852197:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada603e9800. 02000000:00000001:2.0:1788572306.852199:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.852200:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.852201:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60774e00. 00000100:00000001:2.0:1788572306.852204:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.852205:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.852209:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60776a00 x1875449615998464/t219043334273(219043334273) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852217:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.852220:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60776a00 x1875449615998464/t219043334273(219043334273) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852226:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.852228:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.852229:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada50b6ac00. 02000000:00000001:2.0:1788572306.852231:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.852232:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.852233:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=239 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.852236:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.852237:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada603e9000. 02000000:00000001:2.0:1788572306.852239:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.852240:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.852241:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60776a00. 00000100:00000001:2.0:1788572306.852244:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.852245:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.852250:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60741c00 x1875449615998848/t219043334274(219043334274) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852256:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.852259:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60741c00 x1875449615998848/t219043334274(219043334274) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852265:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.852266:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.852268:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4653c400. 02000000:00000001:2.0:1788572306.852271:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.852272:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.852273:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=238 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.852275:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.852276:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51920c00. 02000000:00000001:2.0:1788572306.852278:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.852279:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.852281:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60741c00. 00000100:00000001:2.0:1788572306.852283:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.852284:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.852289:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60740000 x1875449615999232/t219043334275(219043334275) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852295:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.852298:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60740000 x1875449615999232/t219043334275(219043334275) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852304:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.852305:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.852307:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4653fc00. 02000000:00000001:2.0:1788572306.852310:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.852311:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.852312:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=237 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.852314:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.852316:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51921e00. 02000000:00000001:2.0:1788572306.852318:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.852319:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.852320:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60740000. 00000100:00000001:2.0:1788572306.852323:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.852324:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.852329:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60776680 x1875449615999616/t219043334276(219043334276) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852335:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.852338:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60776680 x1875449615999616/t219043334276(219043334276) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852345:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.852347:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.852348:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada50b68000. 02000000:00000001:2.0:1788572306.852350:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.852351:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.852352:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=236 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.852355:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.852356:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada603e9a00. 02000000:00000001:2.0:1788572306.852358:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.852359:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.852361:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60776680. 00000100:00000001:2.0:1788572306.852363:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.852364:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.852369:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60774380 x1875449616000000/t219043334277(219043334277) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852375:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.852378:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60774380 x1875449616000000/t219043334277(219043334277) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852384:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.852386:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.852387:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada50b69c00. 02000000:00000001:2.0:1788572306.852389:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.852390:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.852391:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=235 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.852393:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.852394:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada603e9600. 02000000:00000001:2.0:1788572306.852396:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.852397:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.852398:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60774380. 00000100:00000001:2.0:1788572306.852400:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.852401:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.852405:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60775500 x1875449616000384/t219043334278(219043334278) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852411:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.852414:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60775500 x1875449616000384/t219043334278(219043334278) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852421:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.852422:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.852424:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada51d1fc00. 02000000:00000001:2.0:1788572306.852428:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.852430:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.852431:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=234 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.852433:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.852434:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada603e8000. 02000000:00000001:2.0:1788572306.852436:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.852437:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.852438:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60775500. 00000100:00000001:2.0:1788572306.852441:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.852442:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.852446:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45412a00 x1875449616000768/t219043334279(219043334279) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852454:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.852457:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45412a00 x1875449616000768/t219043334279(219043334279) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852462:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.852464:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.852465:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada42fef400. 02000000:00000001:2.0:1788572306.852490:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.852492:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.852494:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=233 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.852495:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.852497:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51921600. 02000000:00000001:2.0:1788572306.852499:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.852500:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.852501:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45412a00. 00000100:00000001:2.0:1788572306.852504:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.852505:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.852510:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45412680 x1875449616001152/t219043334280(219043334280) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852522:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.852525:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45412680 x1875449616001152/t219043334280(219043334280) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852530:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.852532:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.852533:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada42fefc00. 02000000:00000001:2.0:1788572306.852535:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.852536:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.852538:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=232 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.852539:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.852541:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51920200. 02000000:00000001:2.0:1788572306.852543:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.852543:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.852545:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45412680. 00000100:00000001:2.0:1788572306.852547:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.852548:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.852552:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45410380 x1875449616001536/t219043334281(219043334281) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852558:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.852561:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45410380 x1875449616001536/t219043334281(219043334281) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852567:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.852568:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.852569:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada42fecc00. 02000000:00000001:2.0:1788572306.852572:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.852572:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.852573:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=231 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.852575:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.852576:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51921a00. 02000000:00000001:2.0:1788572306.852578:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.852579:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.852581:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45410380. 00000100:00000001:2.0:1788572306.852583:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.852584:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.852589:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45411500 x1875449616001920/t219043334282(219043334282) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852594:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.852597:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45411500 x1875449616001920/t219043334282(219043334282) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852602:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.852603:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.852604:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada42fee000. 02000000:00000001:2.0:1788572306.852607:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.852607:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.852608:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=230 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.852610:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.852611:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51921000. 02000000:00000001:2.0:1788572306.852613:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.852614:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.852616:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45411500. 00000100:00000001:2.0:1788572306.852618:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.852619:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.852624:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45413100 x1875449616002304/t219043334283(219043334283) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852630:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.852633:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45413100 x1875449616002304/t219043334283(219043334283) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852638:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.852640:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.852641:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada42fef800. 02000000:00000001:2.0:1788572306.852643:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.852644:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.852645:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=229 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.852647:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.852649:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51920000. 02000000:00000001:2.0:1788572306.852650:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.852651:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.852653:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45413100. 00000100:00000001:2.0:1788572306.852655:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.852656:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.852660:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45410700 x1875449616002688/t219043334284(219043334284) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852666:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.852669:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45410700 x1875449616002688/t219043334284(219043334284) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852673:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.852675:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.852676:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada42fed000. 02000000:00000001:2.0:1788572306.852678:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.852679:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.852680:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=228 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.852682:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.852683:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51920600. 02000000:00000001:2.0:1788572306.852685:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.852686:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.852688:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45410700. 00000100:00000001:2.0:1788572306.852690:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.852691:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.852696:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45411180 x1875449616003072/t219043334285(219043334285) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852703:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.852706:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45411180 x1875449616003072/t219043334285(219043334285) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852711:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.852712:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.852713:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada42fef000. 02000000:00000001:2.0:1788572306.852715:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.852717:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.852718:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=227 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.852720:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.852721:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51921400. 02000000:00000001:2.0:1788572306.852723:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.852724:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.852725:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45411180. 00000100:00000001:2.0:1788572306.852726:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.852727:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.852732:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60777100 x1875449616003456/t219043334286(219043334286) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852736:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.852739:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60777100 x1875449616003456/t219043334286(219043334286) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852744:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.852745:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.852746:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada50b68c00. 02000000:00000001:2.0:1788572306.852748:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.852749:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.852750:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=226 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.852751:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.852752:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada603e8200. 02000000:00000001:2.0:1788572306.852754:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.852755:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.852755:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60777100. 00000100:00000001:2.0:1788572306.852757:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.852760:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.852764:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60774700 x1875449616003840/t219043334287(219043334287) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852769:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.852771:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60774700 x1875449616003840/t219043334287(219043334287) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852776:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.852778:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.852779:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada50b6a400. 02000000:00000001:2.0:1788572306.852781:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.852782:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.852783:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=225 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.852784:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.852785:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada603e8800. 02000000:00000001:2.0:1788572306.852787:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.852788:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.852788:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60774700. 00000100:00000001:2.0:1788572306.852790:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.852791:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.852795:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45411880 x1875449616004224/t219043334288(219043334288) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852799:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.852802:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45411880 x1875449616004224/t219043334288(219043334288) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852806:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.852808:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.852809:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada42fec800. 02000000:00000001:2.0:1788572306.852810:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.852812:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.852813:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=224 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.852815:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.852816:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51921c00. 02000000:00000001:2.0:1788572306.852818:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.852819:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.852820:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45411880. 00000100:00000001:2.0:1788572306.852822:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.852823:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.852827:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45413480 x1875449616004608/t219043334289(219043334289) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852832:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.852835:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45413480 x1875449616004608/t219043334289(219043334289) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852841:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.852843:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.852844:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada42fee400. 02000000:00000001:2.0:1788572306.852846:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.852847:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.852848:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=223 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.852849:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.852850:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada5123e800. 02000000:00000001:2.0:1788572306.852887:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.852888:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.852890:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45413480. 00000100:00000001:2.0:1788572306.852892:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.852893:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.852898:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45412300 x1875449616004992/t219043334290(219043334290) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852905:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.852908:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45412300 x1875449616004992/t219043334290(219043334290) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852913:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.852914:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.852916:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada501df400. 02000000:00000001:2.0:1788572306.852920:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.852921:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.852922:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=222 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.852924:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.852925:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada607c9c00. 02000000:00000001:2.0:1788572306.852927:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.852929:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.852930:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45412300. 00000100:00000001:2.0:1788572306.852932:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.852933:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.852937:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45413b80 x1875449616005376/t219043334291(219043334291) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852948:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.852951:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45413b80 x1875449616005376/t219043334291(219043334291) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852956:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.852958:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.852959:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada517e7400. 02000000:00000001:2.0:1788572306.852962:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.852962:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.852964:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=221 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.852965:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.852967:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada607c9600. 02000000:00000001:2.0:1788572306.852969:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.852970:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.852971:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45413b80. 00000100:00000001:2.0:1788572306.852973:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.852974:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.852980:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45411c00 x1875449616005760/t219043334292(219043334292) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852985:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.852988:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45411c00 x1875449616005760/t219043334292(219043334292) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.852993:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.852994:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.852996:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada44dd5000. 02000000:00000001:2.0:1788572306.852999:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.853000:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.853001:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=220 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.853003:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.853004:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada607c8000. 02000000:00000001:2.0:1788572306.853006:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.853007:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.853008:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45411c00. 00000100:00000001:2.0:1788572306.853011:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.853012:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.853016:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60775180 x1875449616006144/t219043334293(219043334293) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853021:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.853024:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60775180 x1875449616006144/t219043334293(219043334293) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853030:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.853031:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.853032:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada50b6a000. 02000000:00000001:2.0:1788572306.853035:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.853036:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.853037:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=219 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.853053:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.853054:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada603e9c00. 02000000:00000001:2.0:1788572306.853056:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.853057:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.853058:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60775180. 00000100:00000001:2.0:1788572306.853060:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.853061:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.853067:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60775880 x1875449616006528/t219043334294(219043334294) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853073:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.853075:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60775880 x1875449616006528/t219043334294(219043334294) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853080:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.853081:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.853082:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada50b69400. 02000000:00000001:2.0:1788572306.853084:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.853085:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.853086:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=218 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.853087:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.853089:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada603e9400. 02000000:00000001:2.0:1788572306.853091:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.853092:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.853093:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60775880. 00000100:00000001:2.0:1788572306.853095:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.853096:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.853101:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60777480 x1875449616006912/t219043334295(219043334295) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853106:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.853109:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60777480 x1875449616006912/t219043334295(219043334295) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853113:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.853115:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.853117:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada50b6a800. 02000000:00000001:2.0:1788572306.853119:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.853120:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.853121:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=217 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.853123:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.853124:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada603e8a00. 02000000:00000001:2.0:1788572306.853126:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.853127:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.853127:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60777480. 00000100:00000001:2.0:1788572306.853130:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.853131:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.853135:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60776d80 x1875449616007296/t219043334296(219043334296) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853142:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.853145:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60776d80 x1875449616007296/t219043334296(219043334296) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853151:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.853152:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.853153:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada50b6b400. 02000000:00000001:2.0:1788572306.853155:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.853157:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.853158:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=216 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.853160:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.853161:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada45481600. 02000000:00000001:2.0:1788572306.853163:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.853164:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.853165:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60776d80. 00000100:00000001:2.0:1788572306.853167:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.853168:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.853174:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60776300 x1875449616007680/t219043334297(219043334297) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853179:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.853181:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60776300 x1875449616007680/t219043334297(219043334297) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853187:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.853188:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.853190:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada435e0400. 02000000:00000001:2.0:1788572306.853193:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.853193:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.853195:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=215 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.853197:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.853198:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada45480e00. 02000000:00000001:2.0:1788572306.853200:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.853201:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.853202:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60776300. 00000100:00000001:2.0:1788572306.853204:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.853205:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.853210:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60777b80 x1875449616008064/t219043334298(219043334298) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853216:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.853218:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60777b80 x1875449616008064/t219043334298(219043334298) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853223:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.853225:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.853226:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada5132e800. 02000000:00000001:2.0:1788572306.853229:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.853230:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.853231:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=214 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.853233:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.853234:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada45480200. 02000000:00000001:2.0:1788572306.853235:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.853236:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.853237:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60777b80. 00000100:00000001:2.0:1788572306.853239:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.853240:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.853244:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60544000 x1875449616008448/t219043334299(219043334299) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853249:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.853252:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60544000 x1875449616008448/t219043334299(219043334299) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853256:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.853257:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.853258:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada44dd7800. 02000000:00000001:2.0:1788572306.853261:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.853261:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.853262:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=213 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.853264:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.853265:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada517d5800. 02000000:00000001:2.0:1788572306.853267:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.853267:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.853269:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60544000. 00000100:00000001:2.0:1788572306.853270:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.853272:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.853276:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60775c00 x1875449616008832/t219043334300(219043334300) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853281:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.853283:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60775c00 x1875449616008832/t219043334300(219043334300) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853289:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.853290:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.853292:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada5132c400. 02000000:00000001:2.0:1788572306.853294:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.853295:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.853296:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=212 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.853298:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.853299:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada45480800. 02000000:00000001:2.0:1788572306.853301:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.853302:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.853303:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60775c00. 00000100:00000001:2.0:1788572306.853305:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.853306:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.853310:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60774000 x1875449616009216/t219043334301(219043334301) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853316:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.853318:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60774000 x1875449616009216/t219043334301(219043334301) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853324:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.853325:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.853326:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada5132f800. 02000000:00000001:2.0:1788572306.853329:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.853330:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.853331:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=211 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.853332:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.853334:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada45480600. 02000000:00000001:2.0:1788572306.853336:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.853337:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.853338:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60774000. 00000100:00000001:2.0:1788572306.853340:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.853341:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.853345:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60544e00 x1875449616009600/t219043334302(219043334302) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853357:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.853360:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60544e00 x1875449616009600/t219043334302(219043334302) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853366:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.853367:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.853368:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada43582000. 02000000:00000001:2.0:1788572306.853371:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.853373:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.853374:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=210 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.853376:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.853377:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada517d5a00. 02000000:00000001:2.0:1788572306.853379:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.853380:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.853381:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60544e00. 00000100:00000001:2.0:1788572306.853383:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.853385:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.853389:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60777800 x1875449616009984/t219043334303(219043334303) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853395:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.853398:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60777800 x1875449616009984/t219043334303(219043334303) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853404:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.853405:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.853407:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada42f1c400. 02000000:00000001:2.0:1788572306.853410:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.853411:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.853412:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=209 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.853414:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.853415:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada45480a00. 02000000:00000001:2.0:1788572306.853417:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.853418:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.853420:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60777800. 00000100:00000001:2.0:1788572306.853422:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.853423:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.853428:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada454cdf80 x1875449616010368/t219043334304(219043334304) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853434:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.853437:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada454cdf80 x1875449616010368/t219043334304(219043334304) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853442:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.853444:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.853445:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada42f1c800. 02000000:00000001:2.0:1788572306.853447:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.853448:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.853449:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=208 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.853451:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.853452:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada45481200. 02000000:00000001:2.0:1788572306.853454:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.853455:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.853456:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada454cdf80. 00000100:00000001:2.0:1788572306.853460:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.853462:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.853467:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada454cce00 x1875449616010752/t219043334305(219043334305) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853474:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.853477:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada454cce00 x1875449616010752/t219043334305(219043334305) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853483:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.853484:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.853485:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada42f1f000. 02000000:00000001:2.0:1788572306.853488:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.853489:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.853490:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=207 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.853492:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.853494:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada45481e00. 02000000:00000001:2.0:1788572306.853496:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.853497:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.853498:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada454cce00. 00000100:00000001:2.0:1788572306.853501:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.853502:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.853506:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada454cea00 x1875449616011136/t219043334306(219043334306) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853513:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.853516:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada454cea00 x1875449616011136/t219043334306(219043334306) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853521:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.853523:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.853524:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada42f1cc00. 02000000:00000001:2.0:1788572306.853527:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.853528:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.853529:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=206 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.853531:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.853532:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada45480c00. 02000000:00000001:2.0:1788572306.853534:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.853535:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.853536:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada454cea00. 00000100:00000001:2.0:1788572306.853539:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.853540:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.853544:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada454ce680 x1875449616011520/t219043334307(219043334307) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853550:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.853553:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada454ce680 x1875449616011520/t219043334307(219043334307) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853558:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.853559:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.853561:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada42f1d800. 02000000:00000001:2.0:1788572306.853564:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.853566:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.853567:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=205 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.853569:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.853570:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada45480400. 02000000:00000001:2.0:1788572306.853572:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.853573:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.853575:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada454ce680. 00000100:00000001:2.0:1788572306.853576:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.853578:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.853582:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada454cc380 x1875449616011904/t219043334308(219043334308) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853589:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.853591:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada454cc380 x1875449616011904/t219043334308(219043334308) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853598:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.853599:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.853600:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada42f1e800. 02000000:00000001:2.0:1788572306.853603:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.853604:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.853605:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=204 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.853608:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.853609:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada45481800. 02000000:00000001:2.0:1788572306.853611:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.853612:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.853613:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada454cc380. 00000100:00000001:2.0:1788572306.853615:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.853616:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.853621:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada454cd500 x1875449616012288/t219043334309(219043334309) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853627:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.853629:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada454cd500 x1875449616012288/t219043334309(219043334309) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853634:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.853636:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.853638:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada42f1f400. 02000000:00000001:2.0:1788572306.853640:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.853641:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.853643:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=203 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.853644:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.853645:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada45481400. 02000000:00000001:2.0:1788572306.853647:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.853648:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.853649:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada454cd500. 00000100:00000001:2.0:1788572306.853652:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.853653:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.853657:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada454cf100 x1875449616012672/t219043334310(219043334310) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853663:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.853666:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada454cf100 x1875449616012672/t219043334310(219043334310) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853672:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.853673:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.853675:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada42f1e400. 02000000:00000001:2.0:1788572306.853678:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.853679:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.853680:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=202 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.853682:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.853683:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada45481000. 02000000:00000001:2.0:1788572306.853685:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.853686:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.853687:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada454cf100. 00000100:00000001:2.0:1788572306.853690:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.853691:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.853696:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45410000 x1875449616013056/t219043334311(219043334311) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853702:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.853704:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45410000 x1875449616013056/t219043334311(219043334311) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853710:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.853711:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.853712:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada517e5000. 02000000:00000001:2.0:1788572306.853714:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.853716:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.853717:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=201 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.853718:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.853719:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada607c8e00. 02000000:00000001:2.0:1788572306.853721:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.853722:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.853723:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45410000. 00000100:00000001:2.0:1788572306.853725:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.853726:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.853730:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45413800 x1875449616013440/t219043334312(219043334312) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853737:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.853740:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45413800 x1875449616013440/t219043334312(219043334312) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853746:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.853747:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.853748:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada517e4c00. 02000000:00000001:2.0:1788572306.853751:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.853752:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.853753:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=200 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.853755:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.853756:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada607c8200. 02000000:00000001:2.0:1788572306.853758:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.853759:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.853760:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45413800. 00000100:00000001:2.0:1788572306.853764:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.853765:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.853769:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45410a80 x1875449616013824/t219043334313(219043334313) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853779:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.853781:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45410a80 x1875449616013824/t219043334313(219043334313) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853787:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.853788:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.853789:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada517e4400. 02000000:00000001:2.0:1788572306.853791:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.853792:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.853793:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=199 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.853795:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.853796:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada607c8800. 02000000:00000001:2.0:1788572306.853798:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.853799:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.853800:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45410a80. 00000100:00000001:2.0:1788572306.853802:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.853803:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.853808:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45411f80 x1875449616014208/t219043334314(219043334314) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853814:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.853817:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45411f80 x1875449616014208/t219043334314(219043334314) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853823:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.853824:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.853826:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada517e7000. 02000000:00000001:2.0:1788572306.853827:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.853829:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.853830:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=198 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.853831:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.853833:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada607c8600. 02000000:00000001:2.0:1788572306.853835:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.853836:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.853838:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45411f80. 00000100:00000001:2.0:1788572306.853841:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.853842:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.853847:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45410e00 x1875449616014592/t219043334315(219043334315) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853894:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.853897:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45410e00 x1875449616014592/t219043334315(219043334315) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853904:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.853905:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.853907:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada517e4000. 02000000:00000001:2.0:1788572306.853910:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.853911:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.853912:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=197 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.853914:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.853915:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada607c8a00. 02000000:00000001:2.0:1788572306.853917:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.853918:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.853919:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45410e00. 00000100:00000001:2.0:1788572306.853921:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.853922:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.853927:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45412d80 x1875449616014976/t219043334316(219043334316) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853933:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.853935:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45412d80 x1875449616014976/t219043334316(219043334316) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853940:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.853941:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.853943:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada517e5800. 02000000:00000001:2.0:1788572306.853944:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.853945:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.853947:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=196 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.853948:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.853950:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada607c9200. 02000000:00000001:2.0:1788572306.853952:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.853953:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.853954:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45412d80. 00000100:00000001:2.0:1788572306.853957:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.853958:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.853962:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada5164b800 x1875449616015360/t219043334317(219043334317) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853968:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.853971:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada5164b800 x1875449616015360/t219043334317(219043334317) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.853977:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.853978:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.853980:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada517e6400. 02000000:00000001:2.0:1788572306.853983:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.853984:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.853985:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=195 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.853987:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.853988:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada607c9e00. 02000000:00000001:2.0:1788572306.853990:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.853991:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.853992:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada5164b800. 00000100:00000001:2.0:1788572306.853995:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.853996:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.854000:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada454cd180 x1875449616015744/t219043334318(219043334318) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.854006:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.854009:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada454cd180 x1875449616015744/t219043334318(219043334318) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.854014:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.854015:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.854017:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada42f1ec00. 02000000:00000001:2.0:1788572306.854019:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.854020:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.854021:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=194 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.854023:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.854024:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada45481c00. 02000000:00000001:2.0:1788572306.854026:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.854027:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.854028:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada454cd180. 00000100:00000001:2.0:1788572306.854030:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.854031:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.854036:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada454cd880 x1875449616016128/t219043334319(219043334319) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.860816:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.860834:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada454cd880 x1875449616016128/t219043334319(219043334319) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.860844:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.860847:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.860866:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada42f1dc00. 02000000:00000001:2.0:1788572306.860875:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.860878:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.860880:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=193 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.860884:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.860887:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada45481a00. 02000000:00000001:2.0:1788572306.860891:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.860894:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.860896:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada454cd880. 00000100:00000001:2.0:1788572306.860901:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.860903:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.860910:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada454cf480 x1875449616016512/t219043334320(219043334320) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.860917:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.860920:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada454cf480 x1875449616016512/t219043334320(219043334320) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.860926:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.860929:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.860931:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada42f1d400. 02000000:00000001:2.0:1788572306.860933:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.860935:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.860936:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=192 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.860938:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.860939:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada603e8e00. 02000000:00000001:2.0:1788572306.860942:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.860943:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.860945:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada454cf480. 00000100:00000001:2.0:1788572306.860947:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.860948:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.860953:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada454ced80 x1875449616016896/t219043334321(219043334321) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.860960:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.860963:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada454ced80 x1875449616016896/t219043334321(219043334321) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.860969:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.860970:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.860972:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada42f1fc00. 02000000:00000001:2.0:1788572306.860975:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.860976:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.860976:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=191 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.860978:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.860980:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51718e00. 02000000:00000001:2.0:1788572306.861003:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.861004:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.861006:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada454ced80. 00000100:00000001:2.0:1788572306.861009:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.861024:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.861031:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada454ce300 x1875449616017280/t219043334322(219043334322) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861038:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.861052:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada454ce300 x1875449616017280/t219043334322(219043334322) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861059:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.861061:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.861063:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada5364b400. 02000000:00000001:2.0:1788572306.861066:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.861068:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.861069:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=190 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.861072:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.861073:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51718800. 02000000:00000001:2.0:1788572306.861075:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.861076:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.861078:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada454ce300. 00000100:00000001:2.0:1788572306.861080:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.861081:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.861085:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada454cfb80 x1875449616017664/t219043334323(219043334323) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861092:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.861094:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada454cfb80 x1875449616017664/t219043334323(219043334323) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861100:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.861101:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.861102:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada53649800. 02000000:00000001:2.0:1788572306.861105:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.861106:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.861107:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=189 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.861109:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.861110:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51718600. 02000000:00000001:2.0:1788572306.861112:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.861113:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.861114:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada454cfb80. 00000100:00000001:2.0:1788572306.861116:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.861117:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.861121:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada454cdc00 x1875449616018048/t219043334324(219043334324) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861132:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.861135:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada454cdc00 x1875449616018048/t219043334324(219043334324) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861140:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.861141:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.861142:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada60ae0400. 02000000:00000001:2.0:1788572306.861145:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.861146:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.861147:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=188 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.861148:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.861150:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51718a00. 02000000:00000001:2.0:1788572306.861152:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.861153:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.861154:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada454cdc00. 00000100:00000001:2.0:1788572306.861156:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.861157:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.861162:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada454cc000 x1875449616018432/t219043334325(219043334325) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861167:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.861170:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada454cc000 x1875449616018432/t219043334325(219043334325) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861175:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.861176:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.861177:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada60ae3400. 02000000:00000001:2.0:1788572306.861180:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.861181:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.861182:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=187 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.861183:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.861185:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51719200. 02000000:00000001:2.0:1788572306.861187:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.861188:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.861190:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada454cc000. 00000100:00000001:2.0:1788572306.861192:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.861193:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.861197:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada454cf800 x1875449616018816/t219043334326(219043334326) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861203:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.861205:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada454cf800 x1875449616018816/t219043334326(219043334326) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861210:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.861211:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.861212:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada60ae1c00. 02000000:00000001:2.0:1788572306.861214:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.861215:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.861216:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=186 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.861218:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.861219:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51719e00. 02000000:00000001:2.0:1788572306.861220:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.861221:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.861222:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada454cf800. 00000100:00000001:2.0:1788572306.861224:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.861225:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.861241:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada455a5f80 x1875449616019200/t219043334327(219043334327) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861247:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.861249:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455a5f80 x1875449616019200/t219043334327(219043334327) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861254:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.861255:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.861256:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada512e1800. 02000000:00000001:2.0:1788572306.861259:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.861261:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.861262:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=185 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.861264:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.861266:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada519ff600. 02000000:00000001:2.0:1788572306.861268:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.861269:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.861271:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455a5f80. 00000100:00000001:2.0:1788572306.861275:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.861276:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.861280:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada455a7800 x1875449616019584/t219043334328(219043334328) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861286:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.861288:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455a7800 x1875449616019584/t219043334328(219043334328) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861294:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.861295:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.861296:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada512e0400. 02000000:00000001:2.0:1788572306.861299:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.861300:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.861301:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=184 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.861303:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.861304:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada519fe800. 02000000:00000001:2.0:1788572306.861306:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.861307:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.861308:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455a7800. 00000100:00000001:2.0:1788572306.861310:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.861311:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.861315:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada455a4000 x1875449616019968/t219043334329(219043334329) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861321:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.861323:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455a4000 x1875449616019968/t219043334329(219043334329) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861328:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.861329:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.861331:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada512e2800. 02000000:00000001:2.0:1788572306.861333:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.861335:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.861336:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=183 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.861338:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.861339:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada519ff200. 02000000:00000001:2.0:1788572306.861341:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.861341:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.861342:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455a4000. 00000100:00000001:2.0:1788572306.861344:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.861346:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.861350:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada455a5c00 x1875449616020352/t219043334330(219043334330) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861355:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.861357:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455a5c00 x1875449616020352/t219043334330(219043334330) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861363:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.861365:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.861366:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada512e2c00. 02000000:00000001:2.0:1788572306.861368:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.861369:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.861370:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=182 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.861372:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.861374:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada519ffe00. 02000000:00000001:2.0:1788572306.861375:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.861376:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.861378:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455a5c00. 00000100:00000001:2.0:1788572306.861380:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.861381:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.861385:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada455a6300 x1875449616020736/t219043334331(219043334331) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861391:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.861393:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455a6300 x1875449616020736/t219043334331(219043334331) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861398:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.861400:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.861401:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada512e2400. 02000000:00000001:2.0:1788572306.861404:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.861405:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.861406:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=181 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.861408:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.861409:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada46491200. 02000000:00000001:2.0:1788572306.861411:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.861413:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.861414:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455a6300. 00000100:00000001:2.0:1788572306.861416:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.861417:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.861421:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada454cca80 x1875449616021120/t219043334332(219043334332) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861427:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.861429:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada454cca80 x1875449616021120/t219043334332(219043334332) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861434:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.861435:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.861437:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada60ae3800. 02000000:00000001:2.0:1788572306.861439:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.861440:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.861441:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=180 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.861443:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.861444:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51718400. 02000000:00000001:2.0:1788572306.861446:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.861447:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.861448:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada454cca80. 00000100:00000001:2.0:1788572306.861450:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.861451:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.861456:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada454cc700 x1875449616021504/t219043334333(219043334333) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861462:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.861464:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada454cc700 x1875449616021504/t219043334333(219043334333) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861471:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.861473:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.861474:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada60ae2400. 02000000:00000001:2.0:1788572306.861476:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.861477:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.861478:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=179 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.861480:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.861481:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51719800. 02000000:00000001:2.0:1788572306.861483:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.861484:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.861485:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada454cc700. 00000100:00000001:2.0:1788572306.861487:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.861488:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.861492:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada51649180 x1875449616021888/t219043334334(219043334334) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861498:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.861501:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada51649180 x1875449616021888/t219043334334(219043334334) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861506:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.861507:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.861508:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada517e5c00. 02000000:00000001:2.0:1788572306.861511:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.861512:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.861513:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=178 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.861515:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.861516:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada607c8c00. 02000000:00000001:2.0:1788572306.861518:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.861519:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.861520:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada51649180. 00000100:00000001:2.0:1788572306.861522:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.861524:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.861528:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4558f480 x1875449616022272/t219043334335(219043334335) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861539:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.861542:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4558f480 x1875449616022272/t219043334335(219043334335) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572315 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861547:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.861548:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.861549:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada60ae2c00. 02000000:00000001:2.0:1788572306.861551:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.861552:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.861553:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=177 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.861554:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.861556:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51719000. 02000000:00000001:2.0:1788572306.861558:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.861559:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.861560:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4558f480. 00000100:00000001:2.0:1788572306.861597:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.861598:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.861603:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4558ed80 x1875449616022656/t219043334336(219043334336) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861609:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.861612:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4558ed80 x1875449616022656/t219043334336(219043334336) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861617:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.861619:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.861620:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada60ae0800. 02000000:00000001:2.0:1788572306.861623:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.861624:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.861625:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=176 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.861627:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.861628:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51719c00. 02000000:00000001:2.0:1788572306.861630:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.861631:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.861634:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4558ed80. 00000100:00000001:2.0:1788572306.861636:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.861637:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.861642:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4558e300 x1875449616023040/t219043334337(219043334337) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861648:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.861650:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4558e300 x1875449616023040/t219043334337(219043334337) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861656:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.861669:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.861672:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada60ae1800. 02000000:00000001:2.0:1788572306.861675:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.861676:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.861677:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=175 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.861680:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.861681:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51719a00. 02000000:00000001:2.0:1788572306.861683:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.861684:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.861685:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4558e300. 00000100:00000001:2.0:1788572306.861687:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.861688:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.861692:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4558fb80 x1875449616023424/t219043334338(219043334338) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861698:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.861701:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4558fb80 x1875449616023424/t219043334338(219043334338) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861707:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.861708:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.861709:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada60ae2800. 02000000:00000001:2.0:1788572306.861711:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.861712:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.861713:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=174 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.861715:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.861717:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51719600. 02000000:00000001:2.0:1788572306.861719:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.861720:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.861722:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4558fb80. 00000100:00000001:2.0:1788572306.861724:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.861725:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.861729:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4558dc00 x1875449616023808/t219043334339(219043334339) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861735:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.861738:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4558dc00 x1875449616023808/t219043334339(219043334339) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861743:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.861745:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.861746:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada60ae2000. 02000000:00000001:2.0:1788572306.861748:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.861749:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.861750:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=173 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.861752:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.861754:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51718000. 02000000:00000001:2.0:1788572306.861755:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.861756:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.861758:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4558dc00. 00000100:00000001:2.0:1788572306.861760:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.861762:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.861767:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4558c000 x1875449616024192/t219043334340(219043334340) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861774:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.861777:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4558c000 x1875449616024192/t219043334340(219043334340) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861784:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.861785:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.861786:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada60ae1000. 02000000:00000001:2.0:1788572306.861788:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.861790:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.861791:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=172 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.861793:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.861794:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51718c00. 02000000:00000001:2.0:1788572306.861797:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.861798:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.861799:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4558c000. 00000100:00000001:2.0:1788572306.861801:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.861802:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.861808:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4558f800 x1875449616024576/t219043334341(219043334341) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861814:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.861817:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4558f800 x1875449616024576/t219043334341(219043334341) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861824:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.861825:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.861826:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada60ae3c00. 02000000:00000001:2.0:1788572306.861829:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.861830:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.861831:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=171 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.861833:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.861834:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51718200. 02000000:00000001:2.0:1788572306.861836:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.861837:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.861838:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4558f800. 00000100:00000001:2.0:1788572306.861840:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.861841:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.861846:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4558ca80 x1875449616024960/t219043334342(219043334342) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861884:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.861888:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4558ca80 x1875449616024960/t219043334342(219043334342) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861895:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.861896:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.861898:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada60ae1400. 02000000:00000001:2.0:1788572306.861901:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.861902:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.861903:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=170 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.861905:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.861907:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada45480000. 02000000:00000001:2.0:1788572306.861915:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.861916:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.861918:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4558ca80. 00000100:00000001:2.0:1788572306.861920:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.861921:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.861927:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4558df80 x1875449616025344/t219043334343(219043334343) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861934:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.861937:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4558df80 x1875449616025344/t219043334343(219043334343) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861943:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.861945:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.861946:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada60ae0c00. 02000000:00000001:2.0:1788572306.861950:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.861951:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.861952:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=169 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.861954:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.861955:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada489acc00. 02000000:00000001:2.0:1788572306.861958:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.861960:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.861961:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4558df80. 00000100:00000001:2.0:1788572306.861964:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.861965:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.861970:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada51648700 x1875449616025728/t219043334344(219043334344) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861976:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.861979:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada51648700 x1875449616025728/t219043334344(219043334344) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.861986:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.861987:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.861989:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada517e4800. 02000000:00000001:2.0:1788572306.861991:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.861992:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.861993:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=168 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.861995:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.861996:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada607c8400. 02000000:00000001:2.0:1788572306.861999:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.862000:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.862001:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada51648700. 00000100:00000001:2.0:1788572306.862003:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.862004:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.862008:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada5164b100 x1875449616026112/t219043334345(219043334345) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862014:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.862017:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada5164b100 x1875449616026112/t219043334345(219043334345) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862023:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.862024:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.862026:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada517e6800. 02000000:00000001:2.0:1788572306.862028:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.862029:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.862030:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=167 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.862032:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.862034:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada607c9800. 02000000:00000001:2.0:1788572306.862036:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.862037:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.862048:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada5164b100. 00000100:00000001:2.0:1788572306.862051:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.862052:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.862057:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada51649500 x1875449616026496/t219043334346(219043334346) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862070:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.862073:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada51649500 x1875449616026496/t219043334346(219043334346) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862079:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.862080:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.862082:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada517e5400. 02000000:00000001:2.0:1788572306.862084:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.862085:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.862086:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=166 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.862088:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.862090:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada607c9400. 02000000:00000001:2.0:1788572306.862092:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.862093:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.862094:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada51649500. 00000100:00000001:2.0:1788572306.862096:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.862097:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.862102:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada51648380 x1875449616026880/t219043334347(219043334347) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862109:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.862111:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada51648380 x1875449616026880/t219043334347(219043334347) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862117:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.862118:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.862119:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada517e7c00. 02000000:00000001:2.0:1788572306.862122:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.862123:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.862124:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=165 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.862126:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.862127:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada607c9000. 02000000:00000001:2.0:1788572306.862129:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.862130:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.862132:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada51648380. 00000100:00000001:2.0:1788572306.862134:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.862135:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.862141:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60544a80 x1875449616027264/t219043334348(219043334348) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862148:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.862150:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60544a80 x1875449616027264/t219043334348(219043334348) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862156:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.862157:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.862158:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada43580000. 02000000:00000001:2.0:1788572306.862160:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.862161:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.862162:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=164 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.862165:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.862166:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada517d4c00. 02000000:00000001:2.0:1788572306.862169:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.862170:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.862171:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60544a80. 00000100:00000001:2.0:1788572306.862173:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.862174:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.862179:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4558ce00 x1875449616027648/t219043334349(219043334349) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862185:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.862188:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4558ce00 x1875449616027648/t219043334349(219043334349) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862193:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.862195:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.862196:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada60ae0000. 02000000:00000001:2.0:1788572306.862198:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.862200:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.862201:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=163 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.862203:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.862205:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada489ad800. 02000000:00000001:2.0:1788572306.862208:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.862209:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.862210:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4558ce00. 00000100:00000001:2.0:1788572306.862213:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.862214:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.862219:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4558ea00 x1875449616028032/t219043334350(219043334350) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862226:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.862229:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4558ea00 x1875449616028032/t219043334350(219043334350) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862235:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.862237:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.862238:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada51262400. 02000000:00000001:2.0:1788572306.862241:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.862242:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.862243:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=162 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.862245:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.862247:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada489ad400. 02000000:00000001:2.0:1788572306.862248:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.862249:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.862251:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4558ea00. 00000100:00000001:2.0:1788572306.862253:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.862254:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.862258:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4558e680 x1875449616028416/t219043334351(219043334351) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862265:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.862268:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4558e680 x1875449616028416/t219043334351(219043334351) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862274:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.862275:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.862277:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada51260c00. 02000000:00000001:2.0:1788572306.862280:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.862281:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.862282:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=161 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.862284:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.862286:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada489ad000. 02000000:00000001:2.0:1788572306.862288:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.862289:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.862290:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4558e680. 00000100:00000001:2.0:1788572306.862292:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.862293:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.862298:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4558c380 x1875449616028800/t219043334352(219043334352) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862303:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.862306:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4558c380 x1875449616028800/t219043334352(219043334352) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862311:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.862312:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.862314:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada51262800. 02000000:00000001:2.0:1788572306.862316:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.862317:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.862318:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=160 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.862320:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.862321:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada489adc00. 02000000:00000001:2.0:1788572306.862323:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.862324:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.862325:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4558c380. 00000100:00000001:2.0:1788572306.862327:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.862328:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.862333:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4558d500 x1875449616029184/t219043334353(219043334353) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862339:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.862342:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4558d500 x1875449616029184/t219043334353(219043334353) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862347:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.862348:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.862349:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada43661c00. 02000000:00000001:2.0:1788572306.862352:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.862353:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.862354:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=159 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.862356:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.862357:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada489ada00. 02000000:00000001:2.0:1788572306.862358:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.862359:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.862360:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4558d500. 00000100:00000001:2.0:1788572306.862362:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.862363:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.862367:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4558f100 x1875449616029568/t219043334354(219043334354) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862373:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.862375:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4558f100 x1875449616029568/t219043334354(219043334354) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862381:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.862382:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.862384:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada43662400. 02000000:00000001:2.0:1788572306.862386:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.862387:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.862388:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=158 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.862390:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.862391:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada489ad600. 02000000:00000001:2.0:1788572306.862393:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.862394:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.862396:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4558f100. 00000100:00000001:2.0:1788572306.862398:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.862399:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.862404:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4558c700 x1875449616029952/t219043334355(219043334355) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862409:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.862411:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4558c700 x1875449616029952/t219043334355(219043334355) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862417:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.862419:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.862420:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada43661400. 02000000:00000001:2.0:1788572306.862422:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.862423:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.862424:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=157 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.862426:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.862427:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada489ac000. 02000000:00000001:2.0:1788572306.862429:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.862430:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.862431:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4558c700. 00000100:00000001:2.0:1788572306.862434:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.862435:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.862439:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4558d180 x1875449616030336/t219043334356(219043334356) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862445:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.862448:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4558d180 x1875449616030336/t219043334356(219043334356) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862454:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.862455:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.862457:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada43661000. 02000000:00000001:2.0:1788572306.862459:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.862460:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.862462:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=156 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.862464:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.862465:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada489ace00. 02000000:00000001:2.0:1788572306.862468:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.862469:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.862470:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4558d180. 00000100:00000001:2.0:1788572306.862472:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.862473:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.862477:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4558d880 x1875449616030720/t219043334357(219043334357) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862488:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.862490:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4558d880 x1875449616030720/t219043334357(219043334357) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862496:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.862497:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.862499:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada43663800. 02000000:00000001:2.0:1788572306.862501:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.862502:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.862503:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=155 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.862504:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.862506:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada489ac200. 02000000:00000001:2.0:1788572306.862507:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.862509:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.862510:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4558d880. 00000100:00000001:2.0:1788572306.862513:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.862514:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.862519:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45407b80 x1875449616031104/t219043334358(219043334358) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862525:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.862527:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45407b80 x1875449616031104/t219043334358(219043334358) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862532:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.862534:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.862535:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada43662000. 02000000:00000001:2.0:1788572306.862537:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.862538:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.862539:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=154 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.862541:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.862542:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada489ac800. 02000000:00000001:2.0:1788572306.862545:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.862546:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.862548:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45407b80. 00000100:00000001:2.0:1788572306.862550:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.862552:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.862556:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45405c00 x1875449616031488/t219043334359(219043334359) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862563:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.862566:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45405c00 x1875449616031488/t219043334359(219043334359) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862572:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.862574:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.862575:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada43661800. 02000000:00000001:2.0:1788572306.862577:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.862578:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.862579:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=153 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.862581:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.862582:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada489ac600. 02000000:00000001:2.0:1788572306.862584:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.862585:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.862587:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45405c00. 00000100:00000001:2.0:1788572306.862589:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.862590:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.862595:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada5164a680 x1875449616031872/t219043334360(219043334360) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862601:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.862604:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada5164a680 x1875449616031872/t219043334360(219043334360) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862609:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.862610:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.862611:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada517e6000. 02000000:00000001:2.0:1788572306.862613:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.862615:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.862615:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=152 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.862617:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.862619:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51920800. 02000000:00000001:2.0:1788572306.862622:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.862623:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.862624:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada5164a680. 00000100:00000001:2.0:1788572306.862626:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.862627:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.862631:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada5164aa00 x1875449616032256/t219043334361(219043334361) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862637:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.862640:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada5164aa00 x1875449616032256/t219043334361(219043334361) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862645:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.862646:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.862647:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada517e7800. 02000000:00000001:2.0:1788572306.862649:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.862650:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.862651:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=151 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.862653:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.862654:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454c1400. 02000000:00000001:2.0:1788572306.862656:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.862657:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.862658:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada5164aa00. 00000100:00000001:2.0:1788572306.862660:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.862661:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.862665:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada51648e00 x1875449616032640/t219043334362(219043334362) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862671:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.862673:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada51648e00 x1875449616032640/t219043334362(219043334362) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862679:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.862680:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.862681:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada48fb7000. 02000000:00000001:2.0:1788572306.862684:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.862685:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.862686:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=150 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.862688:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.862689:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454c1c00. 02000000:00000001:2.0:1788572306.862691:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.862692:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.862693:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada51648e00. 00000100:00000001:2.0:1788572306.862695:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.862696:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.862700:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada51649f80 x1875449616033024/t219043334363(219043334363) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862705:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.862708:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada51649f80 x1875449616033024/t219043334363(219043334363) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862713:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.862714:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.862715:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada48fb4400. 02000000:00000001:2.0:1788572306.862717:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.862719:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.862720:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=149 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.862721:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.862722:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454c1a00. 02000000:00000001:2.0:1788572306.862724:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.862725:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.862726:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada51649f80. 00000100:00000001:2.0:1788572306.862728:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.862729:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.862733:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada51648a80 x1875449616033408/t219043334364(219043334364) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862738:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.862740:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada51648a80 x1875449616033408/t219043334364(219043334364) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862746:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.862747:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.862749:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada48fb4800. 02000000:00000001:2.0:1788572306.862750:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.862751:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.862752:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=148 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.862754:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.862755:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454c1600. 02000000:00000001:2.0:1788572306.862756:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.862758:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.862759:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada51648a80. 00000100:00000001:2.0:1788572306.862761:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.862762:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.862766:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada51648000 x1875449616033792/t219043334365(219043334365) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862772:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.862775:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada51648000 x1875449616033792/t219043334365(219043334365) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862780:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.862782:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.862783:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada48fb7c00. 02000000:00000001:2.0:1788572306.862786:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.862787:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.862788:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=147 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.862789:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.862791:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454c0000. 02000000:00000001:2.0:1788572306.862793:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.862794:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.862795:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada51648000. 00000100:00000001:2.0:1788572306.862798:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.862799:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.862804:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada51649c00 x1875449616034176/t219043334366(219043334366) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862810:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.862814:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada51649c00 x1875449616034176/t219043334366(219043334366) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862819:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.862820:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.862822:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada43660c00. 02000000:00000001:2.0:1788572306.862824:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.862826:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.862827:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=146 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.862829:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.862830:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454c0e00. 02000000:00000001:2.0:1788572306.862832:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.862833:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.862834:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada51649c00. 00000100:00000001:2.0:1788572306.862836:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.862837:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.862842:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45407800 x1875449616034560/t219043334367(219043334367) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862847:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.862850:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45407800 x1875449616034560/t219043334367(219043334367) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862901:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.862902:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.862904:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada43660400. 02000000:00000001:2.0:1788572306.862907:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.862908:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.862909:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=145 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.862911:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.862913:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada489aca00. 02000000:00000001:2.0:1788572306.862915:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.862916:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.862917:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45407800. 00000100:00000001:2.0:1788572306.862920:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.862921:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.862925:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45404a80 x1875449616034944/t219043334368(219043334368) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862937:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.862940:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45404a80 x1875449616034944/t219043334368(219043334368) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862945:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.862946:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.862947:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada43662c00. 02000000:00000001:2.0:1788572306.862949:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.862950:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.862951:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=144 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.862953:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.862954:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada489ad200. 02000000:00000001:2.0:1788572306.862957:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.862958:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.862959:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45404a80. 00000100:00000001:2.0:1788572306.862962:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.862963:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.862968:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45405f80 x1875449616035328/t219043334369(219043334369) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862973:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.862976:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45405f80 x1875449616035328/t219043334369(219043334369) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.862982:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.862983:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.862984:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada43663c00. 02000000:00000001:2.0:1788572306.862987:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.862987:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.862988:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=143 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.862990:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.862991:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada489ade00. 02000000:00000001:2.0:1788572306.862993:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.862994:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.862995:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45405f80. 00000100:00000001:2.0:1788572306.862997:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.862998:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.863003:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45404e00 x1875449616035712/t219043334370(219043334370) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863008:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.863010:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45404e00 x1875449616035712/t219043334370(219043334370) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863015:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.863016:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.863017:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada43663400. 02000000:00000001:2.0:1788572306.863018:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.863019:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.863020:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=142 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.863021:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.863022:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51719400. 02000000:00000001:2.0:1788572306.863024:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.863025:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.863026:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45404e00. 00000100:00000001:2.0:1788572306.863028:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.863029:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.863033:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45406a00 x1875449616036096/t219043334371(219043334371) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863050:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.863054:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45406a00 x1875449616036096/t219043334371(219043334371) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863060:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.863061:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.863062:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada43660000. 02000000:00000001:2.0:1788572306.863064:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.863065:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.863066:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=141 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.863067:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.863069:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada488cca00. 02000000:00000001:2.0:1788572306.863072:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.863073:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.863074:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45406a00. 00000100:00000001:2.0:1788572306.863076:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.863077:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.863082:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45406680 x1875449616036480/t219043334372(219043334372) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863088:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.863091:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45406680 x1875449616036480/t219043334372(219043334372) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863096:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.863097:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.863098:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada43663000. 02000000:00000001:2.0:1788572306.863100:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.863102:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.863103:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=140 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.863105:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.863106:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada488cde00. 02000000:00000001:2.0:1788572306.863108:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.863109:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.863110:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45406680. 00000100:00000001:2.0:1788572306.863112:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.863113:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.863118:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45404380 x1875449616036864/t219043334373(219043334373) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863124:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.863126:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45404380 x1875449616036864/t219043334373(219043334373) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863132:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.863134:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.863135:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45431400. 02000000:00000001:2.0:1788572306.863138:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.863139:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.863141:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=139 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.863143:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.863144:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada488ccc00. 02000000:00000001:2.0:1788572306.863146:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.863147:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.863148:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45404380. 00000100:00000001:2.0:1788572306.863150:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.863151:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.863156:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45405500 x1875449616037248/t219043334374(219043334374) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863162:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.863165:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45405500 x1875449616037248/t219043334374(219043334374) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863171:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.863172:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.863173:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45433800. 02000000:00000001:2.0:1788572306.863175:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.863176:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.863177:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=138 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.863179:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.863180:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada488cc400. 02000000:00000001:2.0:1788572306.863182:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.863183:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.863184:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45405500. 00000100:00000001:2.0:1788572306.863186:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.863187:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.863192:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45407100 x1875449616037632/t219043334375(219043334375) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863198:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.863201:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45407100 x1875449616037632/t219043334375(219043334375) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863206:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.863208:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.863210:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45430c00. 02000000:00000001:2.0:1788572306.863212:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.863214:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.863215:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=137 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.863216:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.863218:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada488cd800. 02000000:00000001:2.0:1788572306.863219:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.863220:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.863221:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45407100. 00000100:00000001:2.0:1788572306.863224:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.863225:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.863230:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45404700 x1875449616038016/t219043334376(219043334376) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863236:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.863239:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45404700 x1875449616038016/t219043334376(219043334376) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863245:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.863246:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.863248:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45430800. 02000000:00000001:2.0:1788572306.863250:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.863251:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.863253:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=136 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.863255:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.863256:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada488cd400. 02000000:00000001:2.0:1788572306.863258:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.863260:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.863261:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45404700. 00000100:00000001:2.0:1788572306.863264:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.863265:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.863270:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45405180 x1875449616038400/t219043334377(219043334377) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863277:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.863280:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45405180 x1875449616038400/t219043334377(219043334377) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863286:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.863288:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.863289:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45433000. 02000000:00000001:2.0:1788572306.863291:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.863293:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.863294:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=135 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.863296:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.863297:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada488cd000. 02000000:00000001:2.0:1788572306.863299:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.863300:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.863302:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45405180. 00000100:00000001:2.0:1788572306.863304:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.863305:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.863309:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada5164bb80 x1875449616038784/t219043334378(219043334378) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863316:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.863318:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada5164bb80 x1875449616038784/t219043334378(219043334378) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863325:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.863326:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.863328:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada48fb7800. 02000000:00000001:2.0:1788572306.863330:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.863331:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.863333:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=134 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.863335:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.863337:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454c0800. 02000000:00000001:2.0:1788572306.863339:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.863340:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.863341:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada5164bb80. 00000100:00000001:2.0:1788572306.863344:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.863345:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.863350:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada5164a300 x1875449616039168/t219043334379(219043334379) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863361:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.863364:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada5164a300 x1875449616039168/t219043334379(219043334379) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863369:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.863370:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.863372:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada48fb6400. 02000000:00000001:2.0:1788572306.863374:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.863375:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.863377:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=133 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.863378:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.863380:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454c0600. 02000000:00000001:2.0:1788572306.863382:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.863383:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.863384:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada5164a300. 00000100:00000001:2.0:1788572306.863386:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.863388:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.863392:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada5164ad80 x1875449616039552/t219043334380(219043334380) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863399:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.863402:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada5164ad80 x1875449616039552/t219043334380(219043334380) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863407:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.863409:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.863410:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada48fb4c00. 02000000:00000001:2.0:1788572306.863412:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.863413:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.863414:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=132 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.863416:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.863417:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454c0a00. 02000000:00000001:2.0:1788572306.863419:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.863420:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.863422:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada5164ad80. 00000100:00000001:2.0:1788572306.863424:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.863425:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.863430:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada5164b480 x1875449616039936/t219043334381(219043334381) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863436:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.863439:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada5164b480 x1875449616039936/t219043334381(219043334381) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863444:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.863445:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.863447:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada48fb5000. 02000000:00000001:2.0:1788572306.863449:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.863451:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.863452:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=131 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.863453:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.863455:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454c1200. 02000000:00000001:2.0:1788572306.863456:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.863457:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.863459:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada5164b480. 00000100:00000001:2.0:1788572306.863461:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.863461:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.863466:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada51649880 x1875449616040320/t219043334382(219043334382) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863472:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.863475:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada51649880 x1875449616040320/t219043334382(219043334382) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863480:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.863481:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.863482:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada48fb6c00. 02000000:00000001:2.0:1788572306.863484:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.863485:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.863486:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=130 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.863488:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.863490:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454c1e00. 02000000:00000001:2.0:1788572306.863492:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.863493:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.863495:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada51649880. 00000100:00000001:2.0:1788572306.863513:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.863515:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.863522:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada455a7480 x1875449616040704/t219043334383(219043334383) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863528:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.863531:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455a7480 x1875449616040704/t219043334383(219043334383) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863537:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.863538:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.863540:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada512e1c00. 02000000:00000001:2.0:1788572306.863544:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.863546:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.863547:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=129 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.863549:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.863550:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada46490e00. 02000000:00000001:2.0:1788572306.863552:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.863553:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.863555:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455a7480. 00000100:00000001:2.0:1788572306.863558:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.863559:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.863564:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada455a7b80 x1875449616041088/t219043334384(219043334384) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863571:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.863574:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455a7b80 x1875449616041088/t219043334384(219043334384) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863580:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.863581:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.863583:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada512e0c00. 02000000:00000001:2.0:1788572306.863585:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.863587:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.863588:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=128 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.863590:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.863591:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada46490200. 02000000:00000001:2.0:1788572306.863593:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.863594:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.863595:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455a7b80. 00000100:00000001:2.0:1788572306.863597:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.863598:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.863603:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada455a5880 x1875449616041472/t219043334385(219043334385) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863609:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.863611:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455a5880 x1875449616041472/t219043334385(219043334385) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863617:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.863618:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.863619:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada512e1400. 02000000:00000001:2.0:1788572306.863621:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.863622:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.863623:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=127 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.863625:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.863626:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada46490000. 02000000:00000001:2.0:1788572306.863628:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.863629:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.863630:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455a5880. 00000100:00000001:2.0:1788572306.863633:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.863634:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.863638:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45642a00 x1875449616041856/t219043334386(219043334386) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863644:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.863646:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45642a00 x1875449616041856/t219043334386(219043334386) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863652:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.863653:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.863655:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada48fb5c00. 02000000:00000001:2.0:1788572306.863657:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.863659:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.863660:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=126 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.863662:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.863663:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454c0400. 02000000:00000001:2.0:1788572306.863664:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.863665:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.863667:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45642a00. 00000100:00000001:2.0:1788572306.863670:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.863671:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.863675:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45642680 x1875449616042240/t219043334387(219043334387) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863681:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.863684:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45642680 x1875449616042240/t219043334387(219043334387) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863689:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.863690:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.863692:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada48fb5800. 02000000:00000001:2.0:1788572306.863694:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.863695:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.863696:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=125 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.863698:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.863699:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454c1800. 02000000:00000001:2.0:1788572306.863700:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.863701:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.863703:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45642680. 00000100:00000001:2.0:1788572306.863704:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.863706:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.863710:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45640380 x1875449616042624/t219043334388(219043334388) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863715:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.863718:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45640380 x1875449616042624/t219043334388(219043334388) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863724:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.863725:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.863726:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada48fb6000. 02000000:00000001:2.0:1788572306.863728:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.863729:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.863730:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=124 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.863732:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.863734:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454c0200. 02000000:00000001:2.0:1788572306.863736:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.863737:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.863738:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45640380. 00000100:00000001:2.0:1788572306.863740:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.863741:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.863745:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45641500 x1875449616043008/t219043334389(219043334389) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863753:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.863756:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45641500 x1875449616043008/t219043334389(219043334389) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863762:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.863764:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.863765:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada48fb6800. 02000000:00000001:2.0:1788572306.863768:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.863769:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.863770:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=123 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.863772:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.863774:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454c1000. 02000000:00000001:2.0:1788572306.863776:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.863777:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.863778:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45641500. 00000100:00000001:2.0:1788572306.863781:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.863783:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.863787:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45643100 x1875449616043392/t219043334390(219043334390) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863800:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.863804:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45643100 x1875449616043392/t219043334390(219043334390) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863810:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.863811:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.863813:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada48fb5400. 02000000:00000001:2.0:1788572306.863815:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.863816:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.863817:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=122 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.863818:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.863819:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada607c9a00. 02000000:00000001:2.0:1788572306.863828:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.863830:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.863831:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45643100. 00000100:00000001:2.0:1788572306.863834:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.863835:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.863841:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45640700 x1875449616043776/t219043334391(219043334391) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863848:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.863891:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45640700 x1875449616043776/t219043334391(219043334391) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863899:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.863900:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.863902:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4543e000. 02000000:00000001:2.0:1788572306.863907:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.863908:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.863910:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=121 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.863911:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.863913:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60641800. 02000000:00000001:2.0:1788572306.863915:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.863916:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.863918:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45640700. 00000100:00000001:2.0:1788572306.863921:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.863922:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.863926:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45641180 x1875449616044160/t219043334392(219043334392) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863933:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.863935:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45641180 x1875449616044160/t219043334392(219043334392) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863942:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.863943:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.863945:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4543f400. 02000000:00000001:2.0:1788572306.863947:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.863948:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.863950:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=120 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.863951:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.863953:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60641000. 02000000:00000001:2.0:1788572306.863955:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.863956:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.863957:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45641180. 00000100:00000001:2.0:1788572306.863959:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.863960:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.863964:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45641880 x1875449616044544/t219043334393(219043334393) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863970:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.863973:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45641880 x1875449616044544/t219043334393(219043334393) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.863978:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.863979:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.863981:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4543ec00. 02000000:00000001:2.0:1788572306.863983:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.863984:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.863985:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=119 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.863987:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.863989:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60641c00. 02000000:00000001:2.0:1788572306.863991:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.863992:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.863993:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45641880. 00000100:00000001:2.0:1788572306.863995:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.863996:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.864000:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45643480 x1875449616044928/t219043334394(219043334394) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.864006:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.864009:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45643480 x1875449616044928/t219043334394(219043334394) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.864015:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.864016:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.864018:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4543d800. 02000000:00000001:2.0:1788572306.864020:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.864021:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.864023:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=118 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.864024:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.864026:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60641a00. 02000000:00000001:2.0:1788572306.864027:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.864028:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.864030:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45643480. 00000100:00000001:2.0:1788572306.864032:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.864033:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.864038:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45642d80 x1875449616045312/t219043334395(219043334395) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.869423:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.869439:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45642d80 x1875449616045312/t219043334395(219043334395) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.869447:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.869451:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.869456:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4543c000. 02000000:00000001:2.0:1788572306.869464:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.869469:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.869471:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=117 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.869474:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.869477:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60641600. 02000000:00000001:2.0:1788572306.869481:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.869483:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.869485:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45642d80. 00000100:00000001:2.0:1788572306.869490:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.869491:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.869497:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45642300 x1875449616045696/t219043334396(219043334396) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.869503:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.869506:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45642300 x1875449616045696/t219043334396(219043334396) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.869511:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.869513:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.869514:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4543fc00. 02000000:00000001:2.0:1788572306.869517:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.869518:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.869519:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=116 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.869521:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.869522:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60640000. 02000000:00000001:2.0:1788572306.869525:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.869526:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.869527:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45642300. 00000100:00000001:2.0:1788572306.869529:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.869530:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.869535:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45643b80 x1875449616046080/t219043334397(219043334397) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.869541:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.869544:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45643b80 x1875449616046080/t219043334397(219043334397) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.869549:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.869551:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.869553:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4543dc00. 02000000:00000001:2.0:1788572306.869555:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.869557:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.869558:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=115 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.869560:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.869561:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60640e00. 02000000:00000001:2.0:1788572306.869563:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.869564:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.869566:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45643b80. 00000100:00000001:2.0:1788572306.869568:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.869569:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.869573:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45641c00 x1875449616046464/t219043334398(219043334398) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.869580:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.869582:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45641c00 x1875449616046464/t219043334398(219043334398) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.869587:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.869589:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.869590:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4543c400. 02000000:00000001:2.0:1788572306.869592:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.869593:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.869594:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=114 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.869596:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.869597:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60640200. 02000000:00000001:2.0:1788572306.869598:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.869599:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.869600:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45641c00. 00000100:00000001:2.0:1788572306.869603:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.869604:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.869608:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45640000 x1875449616046848/t219043334399(219043334399) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.869614:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.869617:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45640000 x1875449616046848/t219043334399(219043334399) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.869624:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.869625:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.869626:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4543e800. 02000000:00000001:2.0:1788572306.869629:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.869631:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.869632:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=113 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.869633:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.869635:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60640800. 02000000:00000001:2.0:1788572306.869637:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.869638:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.869639:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45640000. 00000100:00000001:2.0:1788572306.869641:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.869643:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.869648:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60545f80 x1875449616047232/t219043334400(219043334400) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.869655:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.869658:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60545f80 x1875449616047232/t219043334400(219043334400) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.869787:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.869789:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.869791:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada44876800. 02000000:00000001:2.0:1788572306.869796:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.869797:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.869798:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=112 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.869800:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.869802:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada517d4800. 02000000:00000001:2.0:1788572306.869805:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.869807:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.869808:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60545f80. 00000100:00000001:2.0:1788572306.869812:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.869813:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.869818:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60545c00 x1875449616047616/t219043334401(219043334401) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.869830:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.869833:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60545c00 x1875449616047616/t219043334401(219043334401) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.869839:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.869840:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.869842:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada44875c00. 02000000:00000001:2.0:1788572306.869844:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.869845:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.869846:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=111 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.869848:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.869849:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada517d4a00. 02000000:00000001:2.0:1788572306.869851:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.869882:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.869884:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60545c00. 00000100:00000001:2.0:1788572306.869907:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.869908:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.869914:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60625880 x1875449616048000/t219043334402(219043334402) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.869921:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.869923:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60625880 x1875449616048000/t219043334402(219043334402) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.869929:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.869931:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.869932:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada44875400. 02000000:00000001:2.0:1788572306.869935:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.869936:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.869938:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=110 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.869940:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.869941:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada517d5000. 02000000:00000001:2.0:1788572306.869943:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.869944:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.869946:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60625880. 00000100:00000001:2.0:1788572306.869949:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.869950:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.869955:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60627480 x1875449616048384/t219043334403(219043334403) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.869962:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.869964:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60627480 x1875449616048384/t219043334403(219043334403) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.869970:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.869971:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.869973:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada44876400. 02000000:00000001:2.0:1788572306.869974:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.869976:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.869977:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=109 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.869978:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.869980:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada4551d400. 02000000:00000001:2.0:1788572306.869982:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.869983:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.869984:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60627480. 00000100:00000001:2.0:1788572306.869986:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.869988:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.869993:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60626d80 x1875449616048768/t219043334404(219043334404) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.869999:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.870001:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60626d80 x1875449616048768/t219043334404(219043334404) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.870006:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.870007:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.870009:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada44874400. 02000000:00000001:2.0:1788572306.870011:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.870012:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.870013:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=108 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.870015:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.870016:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada48e09800. 02000000:00000001:2.0:1788572306.870019:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.870020:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.870021:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60626d80. 00000100:00000001:2.0:1788572306.870023:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.870024:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.870029:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60626300 x1875449616049152/t219043334405(219043334405) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.870035:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.870037:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60626300 x1875449616049152/t219043334405(219043334405) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.870053:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.870055:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.870056:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada60568400. 02000000:00000001:2.0:1788572306.870059:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.870060:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.870061:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=107 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.870063:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.870064:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada48e09600. 02000000:00000001:2.0:1788572306.870066:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.870067:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.870068:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60626300. 00000100:00000001:2.0:1788572306.870070:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.870072:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.870077:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60627b80 x1875449616049536/t219043334406(219043334406) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.870083:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.870086:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60627b80 x1875449616049536/t219043334406(219043334406) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.870092:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.870093:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.870095:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada6056a800. 02000000:00000001:2.0:1788572306.870097:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.870099:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.870100:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=106 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.870102:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.870103:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada517d4e00. 02000000:00000001:2.0:1788572306.870105:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.870106:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.870108:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60627b80. 00000100:00000001:2.0:1788572306.870110:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.870111:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.870115:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60625c00 x1875449616049920/t219043334407(219043334407) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.870121:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.870124:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60625c00 x1875449616049920/t219043334407(219043334407) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.870129:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.870130:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.870131:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada6056a400. 02000000:00000001:2.0:1788572306.870133:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.870134:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.870135:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=105 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.870137:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.870138:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada45574a00. 02000000:00000001:2.0:1788572306.870140:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.870141:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.870142:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60625c00. 00000100:00000001:2.0:1788572306.870144:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.870145:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.870149:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60624000 x1875449616050304/t219043334408(219043334408) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.870155:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.870158:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60624000 x1875449616050304/t219043334408(219043334408) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.870163:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.870165:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.870166:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada60569000. 02000000:00000001:2.0:1788572306.870169:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.870170:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.870171:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=104 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.870173:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.870174:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada45575e00. 02000000:00000001:2.0:1788572306.870176:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.870176:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.870178:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60624000. 00000100:00000001:2.0:1788572306.870180:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.870181:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.870185:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60627800 x1875449616050688/t219043334409(219043334409) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.870192:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.870195:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60627800 x1875449616050688/t219043334409(219043334409) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.870200:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.870201:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.870203:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada60569400. 02000000:00000001:2.0:1788572306.870205:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.870206:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.870207:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=103 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.870209:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.870210:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada45574c00. 02000000:00000001:2.0:1788572306.870212:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.870213:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.870214:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60627800. 00000100:00000001:2.0:1788572306.870216:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.870217:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.870221:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada455a4700 x1875449616051072/t219043334410(219043334410) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.870734:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.870741:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455a4700 x1875449616051072/t219043334410(219043334410) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.870749:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.870750:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.870752:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada512e3800. 02000000:00000001:2.0:1788572306.870755:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.870757:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.870758:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=102 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.870760:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.870761:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada46490600. 02000000:00000001:2.0:1788572306.870763:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.870765:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.870766:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455a4700. 00000100:00000001:2.0:1788572306.870769:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.870770:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.870775:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada455a7100 x1875449616051456/t219043334411(219043334411) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.870781:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.870784:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455a7100 x1875449616051456/t219043334411(219043334411) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.870789:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.870791:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.870792:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada512e3c00. 02000000:00000001:2.0:1788572306.870795:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.870796:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.870797:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=101 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.870799:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.870801:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada46491000. 02000000:00000001:2.0:1788572306.870803:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.870804:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.870805:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455a7100. 00000100:00000001:2.0:1788572306.870808:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.870809:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.870814:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45643800 x1875449616051840/t219043334412(219043334412) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.870826:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.870829:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45643800 x1875449616051840/t219043334412(219043334412) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.870835:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.870836:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.870838:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4543e400. 02000000:00000001:2.0:1788572306.870840:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.870841:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.870842:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=100 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.870844:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.870845:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60640600. 02000000:00000001:2.0:1788572306.870847:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.870848:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.870850:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45643800. 00000100:00000001:2.0:1788572306.870966:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.870968:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.870975:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45640a80 x1875449616052224/t219043334413(219043334413) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.870981:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.870984:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45640a80 x1875449616052224/t219043334413(219043334413) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.870989:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.870991:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.870992:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4543d000. 02000000:00000001:2.0:1788572306.870994:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.870996:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.870997:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=99 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.870998:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.871000:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60640a00. 02000000:00000001:2.0:1788572306.871002:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.871003:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.871005:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45640a80. 00000100:00000001:2.0:1788572306.871007:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.871008:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.871012:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45641f80 x1875449616052608/t219043334414(219043334414) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.871017:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.871020:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45641f80 x1875449616052608/t219043334414(219043334414) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.871024:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.871026:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.871027:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4543d400. 02000000:00000001:2.0:1788572306.871029:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.871030:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.871031:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=98 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.871033:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.871034:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60641200. 02000000:00000001:2.0:1788572306.871036:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.871037:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.871038:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45641f80. 00000100:00000001:2.0:1788572306.871050:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.871051:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.871055:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45640e00 x1875449616052992/t219043334415(219043334415) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.871061:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.871064:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45640e00 x1875449616052992/t219043334415(219043334415) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.871069:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.871070:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.871072:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4543f800. 02000000:00000001:2.0:1788572306.871075:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.871076:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.871077:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=97 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.871079:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.871081:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60641e00. 02000000:00000001:2.0:1788572306.871084:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.871085:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.871086:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45640e00. 00000100:00000001:2.0:1788572306.871088:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.871090:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.871094:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4556a680 x1875449616053376/t219043334416(219043334416) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.871101:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.871104:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4556a680 x1875449616053376/t219043334416(219043334416) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.871111:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.871112:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.871113:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4543cc00. 02000000:00000001:2.0:1788572306.871116:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.871117:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.871118:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=96 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.871120:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.871122:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60640c00. 02000000:00000001:2.0:1788572306.871123:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.871124:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.871125:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4556a680. 00000100:00000001:2.0:1788572306.871129:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.871130:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.871135:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4556aa00 x1875449616053760/t219043334417(219043334417) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.871142:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.871145:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4556aa00 x1875449616053760/t219043334417(219043334417) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.871152:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.871153:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.871155:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4543c800. 02000000:00000001:2.0:1788572306.871157:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.871159:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.871160:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=95 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.871161:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.871163:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60640400. 02000000:00000001:2.0:1788572306.871164:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.871166:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.871167:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4556aa00. 00000100:00000001:2.0:1788572306.871169:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.871170:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.871175:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45568e00 x1875449616054144/t219043334418(219043334418) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.871181:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.871184:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45568e00 x1875449616054144/t219043334418(219043334418) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.871188:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.871190:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.871191:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45433400. 02000000:00000001:2.0:1788572306.871193:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.871194:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.871195:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=94 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.871198:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.871199:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454c0c00. 02000000:00000001:2.0:1788572306.871201:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.871202:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.871203:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45568e00. 00000100:00000001:2.0:1788572306.871206:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.871207:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.871211:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45407480 x1875449616054528/t219043334419(219043334419) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.871218:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.871220:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45407480 x1875449616054528/t219043334419(219043334419) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.871225:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.871526:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.871529:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45432c00. 02000000:00000001:2.0:1788572306.871533:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.871535:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.871537:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=93 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.871539:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.871540:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada488cdc00. 02000000:00000001:2.0:1788572306.871543:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.871544:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.871545:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45407480. 00000100:00000001:2.0:1788572306.871549:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.871550:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.871556:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45406d80 x1875449616054912/t219043334420(219043334420) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.871563:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.871566:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45406d80 x1875449616054912/t219043334420(219043334420) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.871573:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.871574:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.871576:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45431800. 02000000:00000001:2.0:1788572306.871578:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.871580:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.871581:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=92 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.871583:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.871584:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada488cda00. 02000000:00000001:2.0:1788572306.871586:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.871587:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.871588:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45406d80. 00000100:00000001:2.0:1788572306.871591:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.871592:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.871597:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45406300 x1875449616055296/t219043334421(219043334421) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.871603:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.871606:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45406300 x1875449616055296/t219043334421(219043334421) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.871612:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.871613:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.871614:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45430000. 02000000:00000001:2.0:1788572306.871617:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.871618:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.871619:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=91 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.871620:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.871622:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada488cd600. 02000000:00000001:2.0:1788572306.871624:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.871625:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.871626:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45406300. 00000100:00000001:2.0:1788572306.871628:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.871629:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.871634:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45405880 x1875449616055680/t219043334422(219043334422) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.871641:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.871644:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45405880 x1875449616055680/t219043334422(219043334422) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.871649:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.871651:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.871652:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45433c00. 02000000:00000001:2.0:1788572306.871655:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.871656:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.871667:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=90 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.871670:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.871671:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada488cc000. 02000000:00000001:2.0:1788572306.871674:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.871675:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.871676:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45405880. 00000100:00000001:2.0:1788572306.871679:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.871680:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.871685:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45404000 x1875449616056064/t219043334423(219043334423) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.871696:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.871699:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45404000 x1875449616056064/t219043334423(219043334423) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.871705:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.871706:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.871707:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45431c00. 02000000:00000001:2.0:1788572306.871709:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.871711:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.871712:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=89 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.871713:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.871715:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada488cce00. 02000000:00000001:2.0:1788572306.871717:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.871718:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.871719:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45404000. 00000100:00000001:2.0:1788572306.871721:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.871723:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.871727:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4565b800 x1875449616056448/t219043334424(219043334424) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.871733:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.871735:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4565b800 x1875449616056448/t219043334424(219043334424) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.871741:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.871742:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.871744:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45430400. 02000000:00000001:2.0:1788572306.871746:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.871747:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.871748:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=88 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.871750:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.871752:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada488cc200. 02000000:00000001:2.0:1788572306.871754:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.871755:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.871757:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4565b800. 00000100:00000001:2.0:1788572306.871759:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.871761:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.871765:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45658a80 x1875449616056832/t219043334425(219043334425) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.871772:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.871774:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45658a80 x1875449616056832/t219043334425(219043334425) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.871781:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.871782:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.871784:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45432800. 02000000:00000001:2.0:1788572306.871786:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.871787:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.871788:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=87 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.871791:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.871792:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada488cc800. 02000000:00000001:2.0:1788572306.871794:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.871795:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.871797:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45658a80. 00000100:00000001:2.0:1788572306.871799:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.871800:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.871805:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45659f80 x1875449616057216/t219043334426(219043334426) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.871812:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.871815:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45659f80 x1875449616057216/t219043334426(219043334426) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.871821:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.871822:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.871823:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45432400. 02000000:00000001:2.0:1788572306.871825:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.871827:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.871828:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=86 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.871830:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.871832:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada488cc600. 02000000:00000001:2.0:1788572306.871834:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.871835:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.871836:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45659f80. 00000100:00000001:2.0:1788572306.871839:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.871840:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.871845:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45658e00 x1875449616057600/t219043334427(219043334427) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.871881:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.871885:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45658e00 x1875449616057600/t219043334427(219043334427) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572316 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.871891:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.871892:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.871894:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45431000. 02000000:00000001:2.0:1788572306.871897:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.871898:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.871899:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=85 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.871901:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.871902:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada60641400. 02000000:00000001:2.0:1788572306.871904:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.871905:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.871907:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45658e00. 00000100:00000001:2.0:1788572306.871909:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.871910:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.871916:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45569f80 x1875449616057984/t219043334428(219043334428) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.871923:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.871925:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45569f80 x1875449616057984/t219043334428(219043334428) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.871931:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.871932:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.871934:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4543f000. 02000000:00000001:2.0:1788572306.871936:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.871938:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.871939:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=84 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.871941:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.871942:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada606f1800. 02000000:00000001:2.0:1788572306.871945:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.871951:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.871952:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45569f80. 00000100:00000001:2.0:1788572306.871954:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.871956:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.871961:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45568a80 x1875449616058368/t219043334429(219043334429) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.871966:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.871969:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45568a80 x1875449616058368/t219043334429(219043334429) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.871974:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.871975:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.871977:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada60731400. 02000000:00000001:2.0:1788572306.871979:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.871980:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.871981:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=83 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.871983:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.871984:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada606f1400. 02000000:00000001:2.0:1788572306.871986:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.871987:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.871988:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45568a80. 00000100:00000001:2.0:1788572306.871990:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.871991:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.871996:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4556b800 x1875449616058752/t219043334430(219043334430) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872002:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.872004:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4556b800 x1875449616058752/t219043334430(219043334430) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872009:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.872011:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.872013:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada60733800. 02000000:00000001:2.0:1788572306.872016:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.872017:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.872018:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=82 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.872019:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.872021:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada606f1000. 02000000:00000001:2.0:1788572306.872022:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.872023:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.872024:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4556b800. 00000100:00000001:2.0:1788572306.872027:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.872028:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.872033:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada60625f80 x1875449616059136/t219043334431(219043334431) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872048:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.872050:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60625f80 x1875449616059136/t219043334431(219043334431) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872056:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.872057:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.872059:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada60569c00. 02000000:00000001:2.0:1788572306.872061:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.872063:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.872064:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=81 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.872065:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.872067:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada45575400. 02000000:00000001:2.0:1788572306.872069:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.872070:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.872071:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60625f80. 00000100:00000001:2.0:1788572306.872074:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.872075:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.872079:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4565aa00 x1875449616059520/t219043334432(219043334432) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872085:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.872087:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4565aa00 x1875449616059520/t219043334432(219043334432) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872094:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.872095:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.872097:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45432000. 02000000:00000001:2.0:1788572306.872113:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.872114:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.872115:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=80 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.872117:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.872119:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada489ac400. 02000000:00000001:2.0:1788572306.872123:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.872124:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.872125:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4565aa00. 00000100:00000001:2.0:1788572306.872127:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.872128:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.872133:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4565a680 x1875449616059904/t219043334433(219043334433) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872139:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.872142:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4565a680 x1875449616059904/t219043334433(219043334433) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872147:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.872148:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.872149:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada456a3c00. 02000000:00000001:2.0:1788572306.872152:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.872153:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.872154:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=79 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.872156:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.872157:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454d0000. 02000000:00000001:2.0:1788572306.872159:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.872160:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.872162:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4565a680. 00000100:00000001:2.0:1788572306.872164:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.872165:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.872169:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45658380 x1875449616060288/t219043334434(219043334434) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872180:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.872182:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45658380 x1875449616060288/t219043334434(219043334434) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872188:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.872189:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.872191:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada456a1c00. 02000000:00000001:2.0:1788572306.872193:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.872194:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.872195:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=78 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.872197:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.872198:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454d0200. 02000000:00000001:2.0:1788572306.872200:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.872200:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.872201:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45658380. 00000100:00000001:2.0:1788572306.872203:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.872204:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.872208:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45659500 x1875449616060672/t219043334435(219043334435) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872214:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.872217:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45659500 x1875449616060672/t219043334435(219043334435) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872222:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.872224:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.872225:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada456a0400. 02000000:00000001:2.0:1788572306.872353:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.872355:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.872356:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=77 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.872358:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.872360:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454d0800. 02000000:00000001:2.0:1788572306.872362:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.872363:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.872366:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45659500. 00000100:00000001:2.0:1788572306.872369:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.872370:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.872375:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4565b100 x1875449616061056/t219043334436(219043334436) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872381:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.872384:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4565b100 x1875449616061056/t219043334436(219043334436) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872390:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.872391:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.872392:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada456a2800. 02000000:00000001:2.0:1788572306.872395:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.872396:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.872397:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=76 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.872399:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.872400:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454d0600. 02000000:00000001:2.0:1788572306.872402:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.872403:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.872404:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4565b100. 00000100:00000001:2.0:1788572306.872406:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.872407:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.872412:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45658700 x1875449616061440/t219043334437(219043334437) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872419:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.872422:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45658700 x1875449616061440/t219043334437(219043334437) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872427:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.872428:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.872430:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada456a2400. 02000000:00000001:2.0:1788572306.872432:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.872433:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.872435:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=75 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.872437:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.872439:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454d0a00. 02000000:00000001:2.0:1788572306.872441:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.872442:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.872443:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45658700. 00000100:00000001:2.0:1788572306.872445:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.872446:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.872450:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45659180 x1875449616061824/t219043334438(219043334438) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872456:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.872459:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45659180 x1875449616061824/t219043334438(219043334438) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872465:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.872466:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.872468:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada456a1000. 02000000:00000001:2.0:1788572306.872470:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.872471:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.872472:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=74 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.872474:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.872476:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454d1200. 02000000:00000001:2.0:1788572306.872477:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.872478:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.872480:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45659180. 00000100:00000001:2.0:1788572306.872482:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.872483:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.872488:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45659880 x1875449616062208/t219043334439(219043334439) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872494:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.872496:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45659880 x1875449616062208/t219043334439(219043334439) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872502:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.872503:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.872505:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada456a1400. 02000000:00000001:2.0:1788572306.872507:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.872509:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.872510:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=73 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.872512:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.872513:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454d1e00. 02000000:00000001:2.0:1788572306.872515:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.872517:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.872518:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45659880. 00000100:00000001:2.0:1788572306.872520:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.872521:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.872526:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4565b480 x1875449616062592/t219043334440(219043334440) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872533:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.872536:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4565b480 x1875449616062592/t219043334440(219043334440) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872542:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.872543:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.872544:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada456a3800. 02000000:00000001:2.0:1788572306.872547:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.872548:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.872549:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=72 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.872551:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.872552:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454d0c00. 02000000:00000001:2.0:1788572306.872554:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.872555:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.872556:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4565b480. 00000100:00000001:2.0:1788572306.872558:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.872559:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.872563:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4565ad80 x1875449616062976/t219043334441(219043334441) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872568:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.872571:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4565ad80 x1875449616062976/t219043334441(219043334441) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872576:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.872578:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.872579:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada456a0c00. 02000000:00000001:2.0:1788572306.872581:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.872582:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.872583:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=71 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.872584:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.872585:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454d0400. 02000000:00000001:2.0:1788572306.872587:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.872588:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.872589:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4565ad80. 00000100:00000001:2.0:1788572306.872591:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.872592:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.872596:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4565a300 x1875449616063360/t219043334442(219043334442) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872602:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.872604:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4565a300 x1875449616063360/t219043334442(219043334442) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872610:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.872611:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.872612:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada456a0800. 02000000:00000001:2.0:1788572306.872614:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.872615:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.872616:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=70 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.872617:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.872618:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454d1800. 02000000:00000001:2.0:1788572306.872620:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.872621:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.872622:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4565a300. 00000100:00000001:2.0:1788572306.872624:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.872625:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.872629:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4565bb80 x1875449616063744/t219043334443(219043334443) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872634:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.872636:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4565bb80 x1875449616063744/t219043334443(219043334443) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872642:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.872642:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.872644:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada456a3000. 02000000:00000001:2.0:1788572306.872646:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.872646:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.872647:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=69 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.872649:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.872650:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454d1400. 02000000:00000001:2.0:1788572306.872651:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.872652:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.872654:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4565bb80. 00000100:00000001:2.0:1788572306.872666:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.872667:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.872673:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45568000 x1875449616064128/t219043334444(219043334444) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872679:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.872682:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45568000 x1875449616064128/t219043334444(219043334444) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872688:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.872689:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.872690:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada60730c00. 02000000:00000001:2.0:1788572306.872692:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.872693:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.872695:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=68 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.872696:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.872697:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada606f1a00. 02000000:00000001:2.0:1788572306.872699:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.872700:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.872701:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45568000. 00000100:00000001:2.0:1788572306.872703:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.872704:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.872708:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45569c00 x1875449616064512/t219043334445(219043334445) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872720:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.872723:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45569c00 x1875449616064512/t219043334445(219043334445) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872729:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.872730:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.872732:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada60730800. 02000000:00000001:2.0:1788572306.872734:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.872734:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.872735:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=67 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.872737:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.872738:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada606f1600. 02000000:00000001:2.0:1788572306.872740:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.872741:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.872742:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45569c00. 00000100:00000001:2.0:1788572306.872744:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.872745:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.872749:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4556bb80 x1875449616064896/t219043334446(219043334446) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872754:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.872756:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4556bb80 x1875449616064896/t219043334446(219043334446) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872762:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.872763:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.872764:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada60733000. 02000000:00000001:2.0:1788572306.872766:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.872767:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.872768:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=66 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.872769:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.872771:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada606f0000. 02000000:00000001:2.0:1788572306.872772:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.872773:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.872775:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4556bb80. 00000100:00000001:2.0:1788572306.872777:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.872778:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.872782:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4556a300 x1875449616065280/t219043334447(219043334447) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872787:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.872790:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4556a300 x1875449616065280/t219043334447(219043334447) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872796:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.872797:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.872799:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada60732000. 02000000:00000001:2.0:1788572306.872802:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.872803:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.872804:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=65 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.872806:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.872807:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada606f0e00. 02000000:00000001:2.0:1788572306.872809:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.872810:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.872811:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4556a300. 00000100:00000001:2.0:1788572306.872814:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.872815:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.872820:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4556ad80 x1875449616065664/t219043334448(219043334448) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872825:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.872828:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4556ad80 x1875449616065664/t219043334448(219043334448) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872833:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.872834:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.872836:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada60733400. 02000000:00000001:2.0:1788572306.872837:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.872838:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.872839:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=64 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.872841:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.872844:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada606f0200. 02000000:00000001:2.0:1788572306.872846:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.872847:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.872848:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4556ad80. 00000100:00000001:2.0:1788572306.872851:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.872898:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.872904:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4556b480 x1875449616066048/t219043334449(219043334449) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872911:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.872913:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4556b480 x1875449616066048/t219043334449(219043334449) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872919:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.872921:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.872922:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada60732c00. 02000000:00000001:2.0:1788572306.872925:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.872927:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.872928:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=63 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.872929:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.872931:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada606f0800. 02000000:00000001:2.0:1788572306.872933:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.872934:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.872935:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4556b480. 00000100:00000001:2.0:1788572306.872938:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.872939:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.872944:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45569880 x1875449616066432/t219043334450(219043334450) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872952:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.872955:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45569880 x1875449616066432/t219043334450(219043334450) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872961:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.872962:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.872964:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada60731800. 02000000:00000001:2.0:1788572306.872966:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.872968:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.872969:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=62 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.872971:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.872972:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada606f0600. 02000000:00000001:2.0:1788572306.872974:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.872975:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.872976:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45569880. 00000100:00000001:2.0:1788572306.872979:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.872980:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.872985:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada455a5500 x1875449616066816/t219043334451(219043334451) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.872992:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.872995:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455a5500 x1875449616066816/t219043334451(219043334451) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.873001:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.873002:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.873003:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada607bec00. 02000000:00000001:2.0:1788572306.873006:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.873007:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.873008:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=61 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.873010:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.873011:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada46491800. 02000000:00000001:2.0:1788572306.873014:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.873015:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.873016:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455a5500. 00000100:00000001:2.0:1788572306.873021:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.873022:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.873027:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada455a4380 x1875449616067200/t219043334452(219043334452) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.873034:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.873036:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455a4380 x1875449616067200/t219043334452(219043334452) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.873054:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.873056:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.873057:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada607bf400. 02000000:00000001:2.0:1788572306.873060:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.873062:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.873063:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=60 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.873065:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.873066:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada46491c00. 02000000:00000001:2.0:1788572306.873069:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.873070:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.873071:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455a4380. 00000100:00000001:2.0:1788572306.873074:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.873075:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.873080:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada455a5180 x1875449616067584/t219043334453(219043334453) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.873087:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.873090:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455a5180 x1875449616067584/t219043334453(219043334453) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.873095:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.873097:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.873098:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada607be000. 02000000:00000001:2.0:1788572306.873100:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.873102:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.873103:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=59 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.873105:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.873107:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada46490c00. 02000000:00000001:2.0:1788572306.873109:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.873110:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.873112:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455a5180. 00000100:00000001:2.0:1788572306.873114:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.873115:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.873120:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45568700 x1875449616067968/t219043334454(219043334454) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.873126:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.873128:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45568700 x1875449616067968/t219043334454(219043334454) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.873135:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.873136:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.873139:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada60733c00. 02000000:00000001:2.0:1788572306.873141:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.873142:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.873143:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=58 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.873145:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.873147:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada606f1200. 02000000:00000001:2.0:1788572306.873149:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.873151:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.873152:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45568700. 00000100:00000001:2.0:1788572306.873154:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.873155:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.873160:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45659c00 x1875449616068352/t219043334455(219043334455) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.873167:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.873169:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45659c00 x1875449616068352/t219043334455(219043334455) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.873175:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.873177:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.873178:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada456a2000. 02000000:00000001:2.0:1788572306.873181:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.873182:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.873183:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=57 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.873185:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.873186:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454d1000. 02000000:00000001:2.0:1788572306.873189:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.873190:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.873191:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45659c00. 00000100:00000001:2.0:1788572306.873193:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.873194:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.873199:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45658000 x1875449616068736/t219043334456(219043334456) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.873210:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.873213:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45658000 x1875449616068736/t219043334456(219043334456) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.873220:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.873222:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.873223:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada456a3400. 02000000:00000001:2.0:1788572306.873225:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.873226:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.875549:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=56 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.875558:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.875565:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454d1c00. 02000000:00000001:2.0:1788572306.875574:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.875576:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.875578:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45658000. 00000100:00000001:2.0:1788572306.875584:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.875586:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.875603:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45460380 x1875449616069120/t219043334457(219043334457) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.875613:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.875617:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45460380 x1875449616069120/t219043334457(219043334457) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.875624:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.875627:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.875630:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada456a2c00. 02000000:00000001:2.0:1788572306.875633:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.875634:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.875635:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=55 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.875637:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.875638:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454d1a00. 02000000:00000001:2.0:1788572306.875640:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.875641:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.875645:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45460380. 00000100:00000001:2.0:1788572306.875647:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.875648:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.875653:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45461500 x1875449616069504/t219043334458(219043334458) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.878010:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.878024:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45461500 x1875449616069504/t219043334458(219043334458) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.878033:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.878037:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.878061:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada456a1800. 02000000:00000001:2.0:1788572306.878069:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.878072:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.878074:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=54 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.878077:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.878080:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454d1600. 02000000:00000001:2.0:1788572306.878083:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.878085:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.878088:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45461500. 00000100:00000001:2.0:1788572306.878092:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.878094:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.878100:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45463100 x1875449616069888/t219043334459(219043334459) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.878107:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.878110:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45463100 x1875449616069888/t219043334459(219043334459) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.878116:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.878117:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.878119:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada456a0000. 02000000:00000001:2.0:1788572306.878121:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.878122:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.878124:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=53 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.878126:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.878129:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada488cd200. 02000000:00000001:2.0:1788572306.878131:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.878132:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.878133:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45463100. 00000100:00000001:2.0:1788572306.878136:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.878137:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.878142:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45460700 x1875449616070272/t219043334460(219043334460) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.878149:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.878151:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45460700 x1875449616070272/t219043334460(219043334460) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.878156:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.878157:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.878159:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45652000. 02000000:00000001:2.0:1788572306.878162:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.878163:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.878164:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=52 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.878165:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.878167:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada607e9200. 02000000:00000001:2.0:1788572306.878169:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.878170:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.878172:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45460700. 00000100:00000001:2.0:1788572306.878174:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.878175:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.878180:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45569500 x1875449616070656/t219043334461(219043334461) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.878186:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.878189:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45569500 x1875449616070656/t219043334461(219043334461) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.878195:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.878196:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.878198:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada60730400. 02000000:00000001:2.0:1788572306.878200:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.878202:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.878203:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=51 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.878205:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.878206:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada606f1e00. 02000000:00000001:2.0:1788572306.878208:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.878209:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.878210:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45569500. 00000100:00000001:2.0:1788572306.878212:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.878214:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.878218:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45568380 x1875449616071040/t219043334462(219043334462) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.878223:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.878226:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45568380 x1875449616071040/t219043334462(219043334462) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.878376:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.878378:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.878379:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada60732800. 02000000:00000001:2.0:1788572306.878382:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.878383:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.878384:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=50 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.878386:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.878388:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada606f0a00. 02000000:00000001:2.0:1788572306.878390:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.878391:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.878392:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45568380. 00000100:00000001:2.0:1788572306.878394:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.878396:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.878402:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada4556b100 x1875449616071424/t219043334463(219043334463) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.878409:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.878411:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada4556b100 x1875449616071424/t219043334463(219043334463) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.878417:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.878419:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.878420:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada60732400. 02000000:00000001:2.0:1788572306.878422:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.878424:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.878425:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=49 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.878426:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.878428:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada606f1c00. 02000000:00000001:2.0:1788572306.878430:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.878431:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.878433:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada4556b100. 00000100:00000001:2.0:1788572306.878435:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.878436:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.878440:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45569180 x1875449616071808/t219043334464(219043334464) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.878446:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.878449:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45569180 x1875449616071808/t219043334464(219043334464) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.878454:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.878455:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.878457:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada60731000. 02000000:00000001:2.0:1788572306.878459:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.878460:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.878461:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=48 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.878463:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.878464:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada606f0400. 02000000:00000001:2.0:1788572306.878465:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.878466:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.878467:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45569180. 00000100:00000001:2.0:1788572306.878470:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.878471:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.878476:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada455bed80 x1875449616072192/t219043334465(219043334465) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.878481:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.878483:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455bed80 x1875449616072192/t219043334465(219043334465) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.878490:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.878491:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.878493:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada60731c00. 02000000:00000001:2.0:1788572306.878495:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.878496:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.878497:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=47 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.878499:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.878500:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51748a00. 02000000:00000001:2.0:1788572306.878502:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.878504:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.878505:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455bed80. 00000100:00000001:2.0:1788572306.878508:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.878509:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.878513:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada455be300 x1875449616072576/t219043334466(219043334466) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.878520:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.878523:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455be300 x1875449616072576/t219043334466(219043334466) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.878528:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.878529:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.878531:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada60730000. 02000000:00000001:2.0:1788572306.878533:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.878534:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.878535:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=46 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.878537:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.878538:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51749e00. 02000000:00000001:2.0:1788572306.878540:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.878542:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.878543:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455be300. 00000100:00000001:2.0:1788572306.878545:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.878546:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.878550:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada455bfb80 x1875449616072960/t219043334467(219043334467) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.878563:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.878566:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455bfb80 x1875449616072960/t219043334467(219043334467) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.878572:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.878573:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.878574:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4566a800. 02000000:00000001:2.0:1788572306.878577:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.878578:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.878579:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=45 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.878581:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.878582:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51748c00. 02000000:00000001:2.0:1788572306.878584:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.878585:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.878586:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455bfb80. 00000100:00000001:2.0:1788572306.878589:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.878590:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.878594:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada455bdc00 x1875449616073344/t219043334468(219043334468) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.878601:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.878604:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455bdc00 x1875449616073344/t219043334468(219043334468) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.878610:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.878611:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.878612:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4566a400. 02000000:00000001:2.0:1788572306.878614:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.878615:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.878616:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=44 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.878617:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.878619:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51748400. 02000000:00000001:2.0:1788572306.878620:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.878621:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.878623:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455bdc00. 00000100:00000001:2.0:1788572306.878625:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.878626:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.878630:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada455bc000 x1875449616073728/t219043334469(219043334469) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.878636:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.878638:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455bc000 x1875449616073728/t219043334469(219043334469) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.878643:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.878644:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.878646:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45669000. 02000000:00000001:2.0:1788572306.878649:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.878650:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.878651:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=43 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.878653:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.878654:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51749800. 02000000:00000001:2.0:1788572306.878656:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.878657:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.878658:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455bc000. 00000100:00000001:2.0:1788572306.879552:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.879555:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.879566:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada455bf800 x1875449616074112/t219043334470(219043334470) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.879575:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.879578:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455bf800 x1875449616074112/t219043334470(219043334470) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.879584:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.879586:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.879589:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45669400. 02000000:00000001:2.0:1788572306.879593:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.879595:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.879596:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=42 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.879599:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.879602:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51749400. 02000000:00000001:2.0:1788572306.879605:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.879607:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.879609:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455bf800. 00000100:00000001:2.0:1788572306.879612:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.879613:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.879618:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada455bca80 x1875449616074496/t219043334471(219043334471) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.879625:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.879628:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455bca80 x1875449616074496/t219043334471(219043334471) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.879634:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.879636:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.879637:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4566b800. 02000000:00000001:2.0:1788572306.879640:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.879642:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.879643:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=41 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.879645:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.879646:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51749000. 02000000:00000001:2.0:1788572306.879648:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.879649:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.879651:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455bca80. 00000100:00000001:2.0:1788572306.879653:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.879654:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.879726:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada455bdf80 x1875449616074880/t219043334472(219043334472) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.879735:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.879739:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455bdf80 x1875449616074880/t219043334472(219043334472) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.879744:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.879746:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.879748:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45668c00. 02000000:00000001:2.0:1788572306.879750:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.879752:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.879754:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=40 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.879756:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.879757:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51749c00. 02000000:00000001:2.0:1788572306.879759:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.879761:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.879762:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455bdf80. 00000100:00000001:2.0:1788572306.879765:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.879766:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.879770:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada455bce00 x1875449616075264/t219043334473(219043334473) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.879777:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.879780:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455bce00 x1875449616075264/t219043334473(219043334473) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.879787:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.879788:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.879790:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45668800. 02000000:00000001:2.0:1788572306.879792:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.879793:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.879794:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=39 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.879797:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.879798:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51749a00. 02000000:00000001:2.0:1788572306.879800:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.879801:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.879802:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455bce00. 00000100:00000001:2.0:1788572306.879804:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.879805:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.879810:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada455bea00 x1875449616075648/t219043334474(219043334474) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.879816:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.879819:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455bea00 x1875449616075648/t219043334474(219043334474) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.879825:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.879827:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.879829:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4566b000. 02000000:00000001:2.0:1788572306.879831:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.879832:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.879833:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=38 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.879835:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.879836:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51749600. 02000000:00000001:2.0:1788572306.879838:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.879839:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.879840:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455bea00. 00000100:00000001:2.0:1788572306.879842:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.879843:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.879849:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45461180 x1875449616076032/t219043334475(219043334475) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.879918:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.879922:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45461180 x1875449616076032/t219043334475(219043334475) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.879928:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.879929:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.879931:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45653400. 02000000:00000001:2.0:1788572306.879934:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.879935:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.879936:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=37 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.879938:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.879940:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada607e8400. 02000000:00000001:2.0:1788572306.879942:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.879943:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.879944:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45461180. 00000100:00000001:2.0:1788572306.879947:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.879948:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.879953:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada455be680 x1875449616076416/t219043334476(219043334476) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.879960:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.879962:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455be680 x1875449616076416/t219043334476(219043334476) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.879968:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.879970:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.879971:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4566a000. 02000000:00000001:2.0:1788572306.879974:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.879975:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.879977:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=36 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.879979:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.879980:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51748000. 02000000:00000001:2.0:1788572306.879982:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.879983:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.879984:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455be680. 00000100:00000001:2.0:1788572306.879987:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.879988:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.879993:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada455bc380 x1875449616076800/t219043334477(219043334477) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.880000:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.880003:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455bc380 x1875449616076800/t219043334477(219043334477) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.880009:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.880010:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.880012:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4566b400. 02000000:00000001:2.0:1788572306.880014:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.880015:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.880016:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=35 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.880018:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.880019:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51748e00. 02000000:00000001:2.0:1788572306.880021:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.880023:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.880024:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455bc380. 00000100:00000001:2.0:1788572306.880026:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.880027:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.880032:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada455bd500 x1875449616077184/t219043334478(219043334478) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.880054:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.880058:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455bd500 x1875449616077184/t219043334478(219043334478) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.880064:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.880066:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.880067:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4566ac00. 02000000:00000001:2.0:1788572306.880070:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.880071:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.880072:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=34 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.880074:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.880075:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51748200. 02000000:00000001:2.0:1788572306.880077:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.880078:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.880080:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455bd500. 00000100:00000001:2.0:1788572306.880082:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.880084:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.880089:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada455bf100 x1875449616077568/t219043334479(219043334479) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.880095:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.880098:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455bf100 x1875449616077568/t219043334479(219043334479) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.880104:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.880106:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.880107:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45669800. 02000000:00000001:2.0:1788572306.880109:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.880110:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.880112:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=33 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.880114:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.880115:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51748800. 02000000:00000001:2.0:1788572306.880117:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.880119:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.880121:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455bf100. 00000100:00000001:2.0:1788572306.880123:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.880124:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.880128:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada455bc700 x1875449616077952/t219043334480(219043334480) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.880135:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.880138:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455bc700 x1875449616077952/t219043334480(219043334480) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.880143:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.880144:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.880146:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45668000. 02000000:00000001:2.0:1788572306.880148:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.880149:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.880150:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=32 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.880152:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.880153:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51748600. 02000000:00000001:2.0:1788572306.880155:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.880156:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.880157:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455bc700. 00000100:00000001:2.0:1788572306.880159:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.880160:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.880164:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada455bd180 x1875449616078336/t219043334481(219043334481) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.880170:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.880173:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455bd180 x1875449616078336/t219043334481(219043334481) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.880178:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.880179:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.880180:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada4566bc00. 02000000:00000001:2.0:1788572306.880182:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.880183:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.880184:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=31 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.880185:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.880187:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada606f0c00. 02000000:00000001:2.0:1788572306.880189:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.880190:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.880191:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455bd180. 00000100:00000001:2.0:1788572306.880193:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.880194:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.880198:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada455bd880 x1875449616078720/t219043334482(219043334482) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.880204:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.880207:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455bd880 x1875449616078720/t219043334482(219043334482) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.880212:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.880213:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.880215:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45669c00. 02000000:00000001:2.0:1788572306.880217:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.880218:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.880219:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=30 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.880222:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.880223:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51729c00. 02000000:00000001:2.0:1788572306.880225:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.880226:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.882243:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455bd880. 00000100:00000001:2.0:1788572306.882258:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.882260:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.882276:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada455bf480 x1875449616079104/t219043334483(219043334483) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.882286:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.882289:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455bf480 x1875449616079104/t219043334483(219043334483) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.882296:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.882300:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.882303:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45668400. 02000000:00000001:2.0:1788572306.882308:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.882313:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.882315:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=29 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.882318:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.882320:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51729600. 02000000:00000001:2.0:1788572306.882324:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.882326:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.882328:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455bf480. 00000100:00000001:2.0:1788572306.882330:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.882331:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.882335:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada456be680 x1875449616079488/t219043334484(219043334484) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.882340:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.882342:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada456be680 x1875449616079488/t219043334484(219043334484) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.882347:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.882348:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.882349:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada454bb800. 02000000:00000001:2.0:1788572306.882352:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.882353:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.882353:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=28 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.882355:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.882356:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51728000. 02000000:00000001:2.0:1788572306.882358:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.882359:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.882359:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada456be680. 00000100:00000001:2.0:1788572306.882362:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.882362:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.882366:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada456bc380 x1875449616079872/t219043334485(219043334485) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.882371:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.882373:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada456bc380 x1875449616079872/t219043334485(219043334485) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.882378:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.882380:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.882381:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada454b8c00. 02000000:00000001:2.0:1788572306.882384:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.882385:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.882386:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=27 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.882387:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.882388:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51728e00. 02000000:00000001:2.0:1788572306.882389:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.882390:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.882391:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada456bc380. 00000100:00000001:2.0:1788572306.882393:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.882393:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.882397:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45463480 x1875449616080256/t219043334486(219043334486) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.882403:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.882405:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45463480 x1875449616080256/t219043334486(219043334486) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.882410:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.882411:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.882413:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45651800. 02000000:00000001:2.0:1788572306.882415:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.882416:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.882417:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=26 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.882419:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.882421:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada607e9a00. 02000000:00000001:2.0:1788572306.882423:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.882424:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.882425:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45463480. 00000100:00000001:2.0:1788572306.882428:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.882429:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.882434:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45462d80 x1875449616080640/t219043334487(219043334487) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.882439:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.882442:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45462d80 x1875449616080640/t219043334487(219043334487) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.882447:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.882448:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.882449:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45650000. 02000000:00000001:2.0:1788572306.882451:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.882452:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.882453:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=25 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.882455:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.882456:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada607e9600. 02000000:00000001:2.0:1788572306.882458:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.882458:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.882460:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45462d80. 00000100:00000001:2.0:1788572306.882462:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.882463:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.882468:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45462300 x1875449616081024/t219043334488(219043334488) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.882474:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.882476:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45462300 x1875449616081024/t219043334488(219043334488) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.882481:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.882483:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.882484:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45653c00. 02000000:00000001:2.0:1788572306.882487:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.882489:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.882490:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=24 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.882491:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.882493:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada607e8000. 02000000:00000001:2.0:1788572306.882494:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.882496:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.882497:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45462300. 00000100:00000001:2.0:1788572306.882499:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.882500:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.882505:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45463b80 x1875449616081408/t219043334489(219043334489) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.882516:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.882519:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45463b80 x1875449616081408/t219043334489(219043334489) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.882525:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.882526:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.882528:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45651c00. 02000000:00000001:2.0:1788572306.882530:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.882531:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.882532:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=23 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.882534:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.882535:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada607e8e00. 02000000:00000001:2.0:1788572306.882537:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.882538:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.882540:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45463b80. 00000100:00000001:2.0:1788572306.882542:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.882543:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.882548:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45461c00 x1875449616082304/t219043334490(219043334490) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.882554:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.882557:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45461c00 x1875449616082304/t219043334490(219043334490) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.882564:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.882565:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.882567:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45650400. 02000000:00000001:2.0:1788572306.882569:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.882569:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.882571:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=22 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.882572:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.882574:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada607e8200. 02000000:00000001:2.0:1788572306.882576:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.882577:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.882578:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45461c00. 00000100:00000001:2.0:1788572306.882580:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.882581:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.882585:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada456bd500 x1875449616082688/t219043334491(219043334491) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.882590:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.882592:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada456bd500 x1875449616082688/t219043334491(219043334491) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.882596:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.882597:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.882598:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada454b8800. 02000000:00000001:2.0:1788572306.882600:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.882601:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.882601:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=21 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.882603:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.882604:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51728200. 02000000:00000001:2.0:1788572306.882605:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.882606:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.882607:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada456bd500. 00000100:00000001:2.0:1788572306.882609:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.882610:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.882614:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada456bf100 x1875449616083072/t219043334492(219043334492) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.882619:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.882621:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada456bf100 x1875449616083072/t219043334492(219043334492) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.882626:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.882627:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.882628:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada454bb000. 02000000:00000001:2.0:1788572306.882630:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.882631:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.882632:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=20 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.882633:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.882635:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51728800. 02000000:00000001:2.0:1788572306.882637:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.882638:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.882639:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada456bf100. 00000100:00000001:2.0:1788572306.882641:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.882642:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.882647:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada456bc700 x1875449616083456/t219043334493(219043334493) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.882653:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.882656:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada456bc700 x1875449616083456/t219043334493(219043334493) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.882723:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.882725:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.882727:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada454ba000. 02000000:00000001:2.0:1788572306.882730:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.882731:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.882732:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=19 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.882734:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.882735:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51728600. 02000000:00000001:2.0:1788572306.882737:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.882738:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.882739:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada456bc700. 00000100:00000001:2.0:1788572306.882741:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.882742:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.882747:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada456bd180 x1875449616083840/t219043334494(219043334494) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.882753:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.882756:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada456bd180 x1875449616083840/t219043334494(219043334494) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.882761:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.882762:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.882763:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada454bb400. 02000000:00000001:2.0:1788572306.882765:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.882767:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.882768:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=18 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.882769:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.882770:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51728a00. 02000000:00000001:2.0:1788572306.882772:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.882773:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.882774:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada456bd180. 00000100:00000001:2.0:1788572306.882776:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.882777:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.882781:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada456bd880 x1875449616084224/t219043334495(219043334495) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.882787:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.882789:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada456bd880 x1875449616084224/t219043334495(219043334495) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.882795:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.882796:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.882797:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada454bac00. 02000000:00000001:2.0:1788572306.882799:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.882800:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.882802:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=17 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.882804:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.882805:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51729200. 02000000:00000001:2.0:1788572306.882807:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.882808:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.882809:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada456bd880. 00000100:00000001:2.0:1788572306.882811:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.882812:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.882816:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45463800 x1875449616084608/t219043334496(219043334496) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.882823:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.882825:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45463800 x1875449616084608/t219043334496(219043334496) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.882830:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.882832:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.882833:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45652400. 02000000:00000001:2.0:1788572306.882835:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.882836:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.882837:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=16 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.882838:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.882839:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada607e8600. 02000000:00000001:2.0:1788572306.882841:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.882842:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.882843:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45463800. 00000100:00000001:2.0:1788572306.882845:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.882846:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.882850:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45460a80 x1875449616084992/t219043334497(219043334497) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.882880:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.882883:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45460a80 x1875449616084992/t219043334497(219043334497) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.882888:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.882889:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.882891:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45651000. 02000000:00000001:2.0:1788572306.882893:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.882895:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.882896:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=15 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.882898:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.882899:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada607e8a00. 02000000:00000001:2.0:1788572306.882901:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.882902:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.882903:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45460a80. 00000100:00000001:2.0:1788572306.882906:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.882907:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.882911:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45461f80 x1875449616085376/t219043334498(219043334498) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.882916:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.882918:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45461f80 x1875449616085376/t219043334498(219043334498) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.882925:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.882926:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.882928:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45651400. 02000000:00000001:2.0:1788572306.882929:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.882930:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.882932:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=14 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.882934:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.882935:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada607e9c00. 02000000:00000001:2.0:1788572306.882937:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.882938:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.882939:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45461f80. 00000100:00000001:2.0:1788572306.882941:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.882942:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.882946:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45460e00 x1875449616085760/t219043334499(219043334499) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.882952:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.882955:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45460e00 x1875449616085760/t219043334499(219043334499) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.882960:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.882961:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.882962:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45653800. 02000000:00000001:2.0:1788572306.882964:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.882965:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.882966:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=13 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.882968:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.882969:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada607e9000. 02000000:00000001:2.0:1788572306.882971:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.882972:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.882973:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45460e00. 00000100:00000001:2.0:1788572306.882975:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.882976:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.882981:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45462a00 x1875449616086144/t219043334500(219043334500) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.882991:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.882994:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45462a00 x1875449616086144/t219043334500(219043334500) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.883000:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.883001:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.883002:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45650c00. 02000000:00000001:2.0:1788572306.883004:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.883005:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.883006:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=12 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.883008:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.883010:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada607e9e00. 02000000:00000001:2.0:1788572306.883012:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.883013:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.883014:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45462a00. 00000100:00000001:2.0:1788572306.883016:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.883017:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.883022:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada45462680 x1875449616086528/t219043334501(219043334501) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.883029:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.883032:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45462680 x1875449616086528/t219043334501(219043334501) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.883038:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.883050:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.883051:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45650800. 02000000:00000001:2.0:1788572306.883053:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.883055:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.883056:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=11 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.883058:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.883059:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada454d0e00. 02000000:00000001:2.0:1788572306.883062:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.883063:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.883065:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45462680. 00000100:00000001:2.0:1788572306.883067:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.883068:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.883072:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada456bed80 x1875449616086912/t219043334502(219043334502) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.883078:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.883081:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada456bed80 x1875449616086912/t219043334502(219043334502) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.883086:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.883087:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.883088:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada454b8000. 02000000:00000001:2.0:1788572306.883090:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.883091:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.883092:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=10 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.883094:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.883095:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51729e00. 02000000:00000001:2.0:1788572306.883097:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.883098:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.883099:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada456bed80. 00000100:00000001:2.0:1788572306.883100:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.883101:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.883106:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada456be300 x1875449616087296/t219043334503(219043334503) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.883111:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.883113:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada456be300 x1875449616087296/t219043334503(219043334503) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.883119:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.883119:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.883121:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada454bbc00. 02000000:00000001:2.0:1788572306.883122:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.883123:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.883124:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=9 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.883126:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.883127:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51728c00. 02000000:00000001:2.0:1788572306.883129:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.883130:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.883131:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada456be300. 00000100:00000001:2.0:1788572306.883133:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.883133:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.883137:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada456bfb80 x1875449616087680/t219043334504(219043334504) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.883143:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.883146:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada456bfb80 x1875449616087680/t219043334504(219043334504) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.883152:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.883153:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.883154:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada454b9c00. 02000000:00000001:2.0:1788572306.883157:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.883158:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.883159:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=8 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.883160:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.883161:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51728400. 02000000:00000001:2.0:1788572306.883163:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.883164:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.883165:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada456bfb80. 00000100:00000001:2.0:1788572306.883167:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.883168:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.883172:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada456bdc00 x1875449616088064/t219043334505(219043334505) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.883178:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.883180:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada456bdc00 x1875449616088064/t219043334505(219043334505) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.883185:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.883186:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.883188:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada454b8400. 02000000:00000001:2.0:1788572306.883190:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.883192:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.883193:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=7 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.883195:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.883196:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51729800. 02000000:00000001:2.0:1788572306.883198:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.883198:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.883200:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada456bdc00. 00000100:00000001:2.0:1788572306.883202:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.883203:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.883207:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada456bc000 x1875449616088448/t219043334506(219043334506) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.883212:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.883214:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada456bc000 x1875449616088448/t219043334506(219043334506) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.883220:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.883221:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.883222:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada454ba800. 02000000:00000001:2.0:1788572306.883224:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.883225:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.883226:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=6 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.883228:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.883229:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51729400. 02000000:00000001:2.0:1788572306.883230:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.883231:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.883232:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada456bc000. 00000100:00000001:2.0:1788572306.883234:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.883235:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.883240:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada456bf800 x1875449616088832/t219043334507(219043334507) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.883245:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.883248:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada456bf800 x1875449616088832/t219043334507(219043334507) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.883254:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.883256:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.883257:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada454ba400. 02000000:00000001:2.0:1788572306.883266:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.883267:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.883268:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=5 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.883270:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.883271:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada51729000. 02000000:00000001:2.0:1788572306.883273:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.883274:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.883276:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada456bf800. 00000100:00000001:2.0:1788572306.883278:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.883279:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.883284:0:2386:0:(client.c:3182:ptlrpc_free_committed()) @@@ commit (last_committed 219043334508) req@ffff9ada456bca80 x1875449616089216/t219043334508(219043334508) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.883290:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.883293:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada456bca80 x1875449616089216/t219043334508(219043334508) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 504/456 e 0 to 0 dl 1788572317 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'unlinkmany.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.883298:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572306.883299:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572306.883300:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada454b9000. 02000000:00000001:2.0:1788572306.883302:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572306.883303:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572306.883304:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=4 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572306.883305:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572306.883307:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada607e8800. 02000000:00000001:2.0:1788572306.883308:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572306.883309:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572306.883310:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada456bca80. 00000100:00000001:2.0:1788572306.883312:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572306.883313:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572306.883315:0:2386:0:(client.c:3189:ptlrpc_free_committed()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000040:2.0:1788572306.883318:0:2386:0:(client.c:3236:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: normal: skip=1 replay=400/400 committed=0/0 00000100:00000001:2.0:1788572306.883321:0:2386:0:(client.c:3237:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788572306.883323:0:2386:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572306.883330:0:2386:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada60624380 x1875449616089472/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572322 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.883338:0:2386:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788572306.883341:0:2386:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788572306.883343:0:2386:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572306.883348:0:2386:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada60624380 x1875449616089472/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572322 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788572306.883355:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616089472:192.168.202.113@tcp:400:kworker.0 00000400:00000001:3.0:1788572306.883998:0:2386:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788572306.884003:0:2386:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788572306.884008:0:2386:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada43d96000. 00000400:00000200:3.0:1788572306.884017:0:2386:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada43c8f4c8 00000400:00000010:3.0:1788572306.884020:0:2386:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada43c8f4c8. 00000100:00000001:3.0:1788572306.884025:0:2386:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572306.884033:0:2386:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada60627100 x1875449616089728/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572322 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788572306.884054:0:2386:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada60627100 x1875449616089728/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572322 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572306.884062:0:2386:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788572306.884064:0:2386:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788572306.884067:0:2386:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:3.0:1788572306.884070:0:2386:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788572306.884072:0:2386:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788572306.884075:0:2386:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788572306.884078:0:2386:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788572306.884090:0:2386:0:(import.c:2195:obd_at_measure()) add 0 to ffff9ada48407c60 time=151 v=5 (5 5 5 5) 00000100:00001000:3.0:1788572306.884094:0:2386:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada48407c18 time=28 v=5 (1 1 1 1) 00000100:00000001:3.0:1788572306.884096:0:2386:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788572306.884098:0:2386:0:(client.c:1506:ptlrpc_check_status()) Process leaving (rc=18446744073709551509 : -107 : ffffffffffffff95) 00000100:00000001:3.0:1788572306.884102:0:2386:0:(recover.c:202:ptlrpc_request_handle_notconn()) Process entered 00000100:00080000:3.0:1788572306.884105:0:2386:0:(recover.c:205:ptlrpc_request_handle_notconn()) import lustre-OST0000-osc-ffff9ada58b0a000 of lustre-OST0000_UUID@192.168.202.113@tcp abruptly disconnected: reconnecting 00000100:00080000:3.0:1788572306.884111:0:2386:0:(import.c:65:import_set_state_nolock()) ffff9ada48407800 lustre-OST0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:3.0:1788572306.884115:0:2386:0:(import.c:182:ptlrpc_set_import_discon()) lustre-OST0000-osc-ffff9ada58b0a000: Connection to lustre-OST0000 (at 192.168.202.113@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00000001:3.0:1788572306.884119:0:2386:0:(obd_class.h:1222:obd_import_event()) Process entered 00000008:00000001:3.0:1788572306.884123:0:2386:0:(osc_request.c:3902:osc_import_event()) Process entered 00000008:00000001:3.0:1788572306.884126:0:2386:0:(osc_request.c:3969:osc_import_event()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572306.884128:0:2386:0:(obd_class.h:1233:obd_import_event()) Process leaving 00000100:00000001:3.0:1788572306.884130:0:2386:0:(import.c:745:ptlrpc_connect_import_locked()) Process entered 00000100:00080000:3.0:1788572306.884132:0:2386:0:(import.c:65:import_set_state_nolock()) ffff9ada48407800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00000001:3.0:1788572306.884136:0:2386:0:(import.c:520:import_select_connection()) Process entered 00000100:00080000:3.0:1788572306.884140:0:2386:0:(import.c:545:import_select_connection()) lustre-OST0000-osc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 1046 00000400:00000200:3.0:1788572306.884148:0:2386:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572306.884154:0:2386:0:(peer.c:1689:LNetPeerDiscovered()) Peer NID 192.168.202.113@tcp is discovered: rc = 1 00000100:00000040:3.0:1788572306.884158:0:2386:0:(lustre_net.h:1977:ptlrpc_connection_put()) PUT conn=ffff9ada51db1200 refcount 14 to 192.168.202.113@tcp 00000100:00000001:3.0:1788572306.884161:0:2386:0:(connection.c:157:ptlrpc_connection_addref()) Process entered 00000100:00000040:3.0:1788572306.884163:0:2386:0:(connection.c:162:ptlrpc_connection_addref()) conn=ffff9ada51db1200 refcount 15 to 192.168.202.113@tcp 00000100:00000001:3.0:1788572306.884166:0:2386:0:(connection.c:164:ptlrpc_connection_addref()) Process leaving (rc=18446632861199700480 : -111212509851136 : ffff9ada51db1200) 00000020:00000001:3.0:1788572306.884168:0:2386:0:(genops.c:792:class_conn2export()) Process entered 00000020:00000040:3.0:1788572306.884170:0:2386:0:(genops.c:804:class_conn2export()) looking for export cookie 0xef30c340bfb39c83 00000020:00000001:3.0:1788572306.884173:0:2386:0:(lustre_handles.c:130:class_handle2object()) Process entered 00000020:00000040:3.0:1788572306.884176:0:2386:0:(lustre_handles.c:148:class_handle2object()) GET export ffff9ada50c3b800 refcount=4 00000020:00000001:3.0:1788572306.884179:0:2386:0:(lustre_handles.c:155:class_handle2object()) Process leaving (rc=18446632861181392896 : -111212528158720 : ffff9ada50c3b800) 00000020:00000001:3.0:1788572306.884181:0:2386:0:(genops.c:806:class_conn2export()) Process leaving (rc=18446632861181392896 : -111212528158720 : ffff9ada50c3b800) 00000100:00000040:3.0:1788572306.884184:0:2386:0:(lustre_net.h:1977:ptlrpc_connection_put()) PUT conn=ffff9ada51db1200 refcount 14 to 192.168.202.113@tcp 00000100:00000001:3.0:1788572306.884186:0:2386:0:(connection.c:157:ptlrpc_connection_addref()) Process entered 00000100:00000040:3.0:1788572306.884187:0:2386:0:(connection.c:162:ptlrpc_connection_addref()) conn=ffff9ada51db1200 refcount 15 to 192.168.202.113@tcp 00000100:00000001:3.0:1788572306.884189:0:2386:0:(connection.c:164:ptlrpc_connection_addref()) Process leaving (rc=18446632861199700480 : -111212509851136 : ffff9ada51db1200) 00000020:00000040:3.0:1788572306.884192:0:2386:0:(genops.c:879:class_export_put()) PUTting export ffff9ada50c3b800 : new refcount 3 00000100:00080000:3.0:1788572306.884195:0:2386:0:(import.c:665:import_select_connection()) lustre-OST0000-osc-ffff9ada58b0a000: import ffff9ada48407800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00000001:3.0:1788572306.884197:0:2386:0:(import.c:672:import_select_connection()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788572306.884199:0:2386:0:(sec.c:1579:sptlrpc_import_sec_adapt()) Process entered 02000000:00000001:3.0:1788572306.884209:0:2386:0:(sec.c:1619:sptlrpc_import_sec_adapt()) Process leaving via out (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788572306.884212:0:2386:0:(sec.c:1647:sptlrpc_import_sec_adapt()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572306.884214:0:2386:0:(obd_class.h:888:obd_reconnect()) Process entered 00000008:00100000:3.0:1788572306.884218:0:2386:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683394 ocd_grant: 8388608, lost: 0. 00000008:00000001:3.0:1788572306.884221:0:2386:0:(osc_request.c:3833:osc_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572306.884224:0:2386:0:(obd_class.h:902:obd_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788572306.884245:0:2386:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada455a6d80. 00000020:00000040:3.0:1788572306.884249:0:2386:0:(genops.c:1210:class_import_get()) import ffff9ada48407800 refcount=4 obd=lustre-OST0000-osc-ffff9ada58b0a000 02000000:00000001:3.0:1788572306.884254:0:2386:0:(sec.c:2038:sptlrpc_sepol_get()) Process entered 02000000:00000001:3.0:1788572306.884256:0:2386:0:(sec.c:2048:sptlrpc_sepol_get()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572306.884260:0:2386:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788572306.884263:0:2386:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788572306.884268:0:2386:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788572306.884277:0:2386:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 1024 at ffff9ada607bcc00. 00000100:00000001:3.0:1788572306.884284:0:2386:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788572306.884291:0:2386:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada455a6d80 x1875449616089984/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788572306.884300:0:2386:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788572306.884305:0:2386:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572306.884309:0:2386:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000009dc80bdd] to pc [ptlrpcd_rcv+-1] req@ffff9ada455a6d80 x1875449616089984/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'ptlrpcd_01_01.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572306.884371:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788572306.884376:0:2382:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788572306.884379:0:2382:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788572306.884384:0:2382:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada455a6d80 x1875449616089984/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'ptlrpcd_01_01.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572306.884392:0:2382:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788572306.884394:0:2382:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572306.884398:0:2382:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788572306.884401:0:2382:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788572306.884404:0:2382:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788572306.884407:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616089984:192.168.202.113@tcp:8:ptlrpcd_01_01.0 00000100:00000001:0.0:1788572306.884412:0:2382:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788572306.884417:0:2382:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788572306.884419:0:2382:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572306.884423:0:2382:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788572306.884428:0:2382:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at ffff9ada45652c00. 02000000:00000001:0.0:1788572306.884430:0:2382:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788572306.884437:0:2382:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:0.0:1788572306.884441:0:2382:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada60acaf20. 00000400:00000010:0.0:1788572306.884448:0:2382:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada5176a088. 00000100:00000200:0.0:1788572306.884457:0:2382:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1875449616089984, portal 4 00000100:00000001:0.0:1788572306.884460:0:2382:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788572306.884462:0:2382:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860989943168 : -111212719608448 : ffff9ada455a6d80) 00000100:00000040:0.0:1788572306.884469:0:2382:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada455a6d80 x1875449616089984/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572316 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'ptlrpcd_01_01.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572306.884476:0:2382:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788572306.884480:0:2382:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:0.0:1788572306.884482:0:2382:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada5176acc0. 00000100:00000200:0.0:1788572306.884486:0:2382:0:(niobuf.c:94:ptl_send_buf()) Sending 520 bytes to portal 28, xid 1875449616089984, offset 0 00000400:00000200:0.0:1788572306.884494:0:2382:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:0.0:1788572306.884503:0:2382:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:0.0:1788572306.884511:0:2382:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:0.0:1788572306.884518:0:2382:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788572306.884523:0:2382:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:0.0:1788572306.884527:0:2382:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788572306.884530:0:2382:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:0.0:1788572306.884535:0:2382:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22273:22273:256:4294967295] 192.168.202.113@tcp LPNI seq info [22269:22269:8:4294967295] 00000400:00000200:0.0:1788572306.884540:0:2382:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:0.0:1788572306.884550:0:2382:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:0.0:1788572306.884557:0:2382:0:(socklnd_cb.c:931:ksocknal_send()) sending 520 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:0.0:1788572306.884562:0:2382:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada5074a500. 00000800:00000200:0.0:1788572306.884569:0:2382:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:0.0:1788572306.884578:0:2382:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:0.0:1788572306.884582:0:2382:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada5074a500 type 1, nob 616 niov 1 nkiov 1 00000100:00000001:0.0:1788572306.884607:0:2382:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572306.884612:0:2382:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: using a timeout of 18848 usecs (19 jiffies) 00000400:00000200:0.0:1788572306.884615:0:2382:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:0.0:1788572306.884618:0:2382:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298891872 00000100:00000001:0.0:1788572306.884621:0:2382:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298902356 00000100:00000001:0.0:1788572306.884623:0:2382:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298891872 00000100:00000001:0.0:1788572306.884626:0:2382:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298902356 00000100:00000001:0.0:1788572306.884629:0:2382:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788572306.884631:0:2382:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572306.884634:0:2382:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572306.884637:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788572306.888618:0:2386:0:(import.c:877:ptlrpc_connect_import_locked()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572306.888635:0:2386:0:(recover.c:221:ptlrpc_request_handle_notconn()) Process leaving 00000100:00000001:3.0:1788572306.888637:0:2386:0:(client.c:1715:after_reply()) Process leaving (rc=18446744073709551509 : -107 : ffffffffffffff95) 00000100:00000040:3.0:1788572306.888671:0:2386:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada60627100 x1875449616089728/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572322 ref 1 fl Rpc:RNQU/200/0 rc -107/-107 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572306.888683:0:2386:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=18446744073709551509 : -107 : 0xffffffffffffff95) 00000100:00000001:3.0:1788572306.888688:0:2386:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788572306.888690:0:2386:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788572306.888695:0:2386:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada60627100 x1875449616089728/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572322 ref 1 fl Interpret:RNQU/200/0 rc -107/-107 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572306.888704:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616089728:192.168.202.113@tcp:400:kworker.0 00000100:00000001:3.0:1788572306.888709:0:2386:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572306.888713:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572306.888718:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60624380 x1875449616089472/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572322 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572306.888725:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788572306.888728:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788572306.888734:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada45574200. 02000000:00000001:3.0:1788572306.888742:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788572306.888745:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:3.0:1788572306.888747:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=3 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:3.0:1788572306.888750:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:3.0:1788572306.888753:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada428be100. 02000000:00000001:3.0:1788572306.888757:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788572306.888759:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788572306.888761:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60624380. 00000100:00000001:3.0:1788572306.888766:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788572306.888767:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788572306.888768:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572306.888771:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60627100 x1875449616089728/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572322 ref 1 fl Complete:RNQU/200/0 rc -107/-107 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572306.888777:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788572306.888779:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788572306.888780:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada45575c00. 02000000:00000001:3.0:1788572306.888782:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788572306.888783:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:3.0:1788572306.888784:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada48407800 refcount=3 obd=lustre-OST0000-osc-ffff9ada58b0a000 00000020:00000001:3.0:1788572306.888786:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:3.0:1788572306.888787:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada428be300. 02000000:00000001:3.0:1788572306.888789:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788572306.888789:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788572306.888790:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60627100. 00000100:00000001:3.0:1788572306.888792:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788572306.888794:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788572306.888798:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572306.888801:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788572306.888804:0:2386:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788572306.888813:0:2386:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788572306.888815:0:2386:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788572306.888819:0:2386:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada43d96688. 00000400:00000200:3.0:1788572306.888824:0:2386:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada43c8f330 00000400:00000010:3.0:1788572306.888827:0:2386:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada43c8f330. 00000100:00000001:3.0:1788572306.888832:0:2386:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572306.888837:0:2386:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada60626680 x1875449616089344/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572322 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788572306.888871:0:2386:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada60626680 x1875449616089344/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572322 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572306.888878:0:2386:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788572306.888880:0:2386:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788572306.888882:0:2386:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:3.0:1788572306.888886:0:2386:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788572306.888888:0:2386:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788572306.888891:0:2386:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788572306.888894:0:2386:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788572306.888901:0:2386:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada58a62c60 time=68 v=5 (5 5 5 5) 00000100:00001000:3.0:1788572306.888906:0:2386:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada58a62c18 time=98 v=5 (1 0 0 0) 00000100:00000001:3.0:1788572306.888908:0:2386:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788572306.888910:0:2386:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788572306.888914:0:2386:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788572306.888916:0:2386:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572306.888918:0:2386:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572306.888923:0:2386:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada60626680 x1875449616089344/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572322 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572306.888929:0:2386:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788572306.888931:0:2386:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788572306.888932:0:2386:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788572306.888936:0:2386:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada60626680 x1875449616089344/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572322 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572306.888947:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449616089344:192.168.202.113@tcp:400:kworker.0 00000400:00000001:3.0:1788572306.888953:0:2386:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788572306.888954:0:2386:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788572306.888957:0:2386:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada43d967e8. 00000400:00000200:3.0:1788572306.888959:0:2386:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada43c8f880 00000400:00000010:3.0:1788572306.888961:0:2386:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada43c8f880. 00000100:00000001:3.0:1788572306.888963:0:2386:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572306.888967:0:2386:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada60626a00 x1875449616089600/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572322 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788572306.888976:0:2386:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada60626a00 x1875449616089600/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572322 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572306.888982:0:2386:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788572306.888983:0:2386:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788572306.888985:0:2386:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:3.0:1788572306.888986:0:2386:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788572306.888987:0:2386:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788572306.888989:0:2386:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788572306.888990:0:2386:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788572306.888996:0:2386:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada58b08460 time=68 v=5 (5 5 5 5) 00000100:00001000:3.0:1788572306.888999:0:2386:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada58b08418 time=68 v=5 (1 1 1 1) 00000100:00000001:3.0:1788572306.889001:0:2386:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788572306.889002:0:2386:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788572306.889004:0:2386:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788572306.889008:0:2386:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572306.889011:0:2386:0:(client.c:3110:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788572306.889014:0:2386:0:(client.c:3122:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff9ada58b0a000: skip recheck: last_committed 0 00000100:00000001:3.0:1788572306.889015:0:2386:0:(client.c:3123:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788572306.889017:0:2386:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572306.889021:0:2386:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada60626a00 x1875449616089600/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572322 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572306.889028:0:2386:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788572306.889029:0:2386:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788572306.889031:0:2386:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788572306.889034:0:2386:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada60626a00 x1875449616089600/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572322 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572306.889050:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616089600:192.168.202.113@tcp:400:kworker.0 00000100:00000001:3.0:1788572306.889054:0:2386:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788572306.889055:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572306.889058:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60626680 x1875449616089344/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572322 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572306.889068:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788572306.889069:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788572306.889071:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada45575000. 02000000:00000001:3.0:1788572306.889073:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788572306.889075:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:3.0:1788572306.889076:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada58a62800 refcount=3 obd=MGC192.168.202.113@tcp 00000020:00000001:3.0:1788572306.889078:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:3.0:1788572306.889079:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada428be000. 02000000:00000001:3.0:1788572306.889081:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788572306.889082:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788572306.889083:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60626680. 00000100:00000001:3.0:1788572306.889086:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788572306.889088:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788572306.889089:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572306.889092:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60626a00 x1875449616089600/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572322 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572306.889098:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788572306.889099:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788572306.889100:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada45575800. 02000000:00000001:3.0:1788572306.889102:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788572306.889103:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:3.0:1788572306.889104:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada58b08000 refcount=3 obd=lustre-MDT0001-mdc-ffff9ada58b0a000 00000020:00000001:3.0:1788572306.889105:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:3.0:1788572306.889106:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada428be400. 02000000:00000001:3.0:1788572306.889108:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788572306.889109:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788572306.889110:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60626a00. 00000100:00000001:3.0:1788572306.889112:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788572306.889113:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788572306.889114:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788572306.889123:0:2386:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788572306.889124:0:2386:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572306.889127:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788572306.889128:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788572306.889414:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788572306.889420:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada5074a500. 00000400:00000200:2.0:1788572306.889426:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572306.889433:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:2.0:1788572306.889441:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada5176acc0 00000400:00000010:2.0:1788572306.889443:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada5176acc0. 00000100:00000001:2.0:1788572306.889447:0:2377:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:2.0:1788572306.889452:0:2377:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada455a6d80 x1875449616089984/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572316 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'ptlrpcd_01_01.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.889460:0:2377:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572306.889462:0:2377:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada455a6d80 x1875449616089984/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572316 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'ptlrpcd_01_01.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572306.889469:0:2377:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572306.889470:0:2377:0:(events.c:69:request_out_callback()) Process leaving 00000100:00000001:0.0:1788572306.892901:0:2382:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788572306.892912:0:2382:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=5 : 5 : 5) 00000100:00000001:0.0:1788572306.892919:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788572306.892922:0:2382:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788572306.892926:0:2382:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572306.892930:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788572306.914826:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572306.914905:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572306.914912:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572306.914917:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572306.914928:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572306.914938:0:2378:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572306.914949:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572306.914955:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572306.914963:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572306.914967:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 4 MB=0x6a9b624294380 00000400:00000200:3.0:1788572306.914973:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572306.914979:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.202.113@tcp of length 224/224 into md 0x356a5 [1] + 0 00000800:00000001:3.0:1788572306.914987:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572306.914994:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572306.914999:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572306.915004:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572306.915014:0:2378:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572306.915023:0:2378:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada455a6d80 x1875449616089984/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572316 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'ptlrpcd_01_01.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572306.915035:0:2378:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=0 replen=544 req@ffff9ada455a6d80 x1875449616089984/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572316 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'ptlrpcd_01_01.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572306.915057:0:2378:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572306.915068:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572306.915071:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572306.915568:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788572306.915575:0:2382:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788572306.915584:0:2382:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788572306.915586:0:2382:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788572306.915590:0:2382:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada60acaf20. 00000400:00000200:0.0:1788572306.915596:0:2382:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada5176a088 00000400:00000010:0.0:1788572306.915599:0:2382:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada5176a088. 00000100:00000001:0.0:1788572306.915603:0:2382:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:0.0:1788572306.915609:0:2382:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada455a6d80 x1875449616089984/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572316 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'ptlrpcd_01_01.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788572306.915617:0:2382:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada455a6d80 x1875449616089984/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572316 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'ptlrpcd_01_01.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572306.915624:0:2382:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788572306.915626:0:2382:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572306.915629:0:2382:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:0.0:1788572306.915633:0:2382:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788572306.915635:0:2382:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788572306.915637:0:2382:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572306.915641:0:2382:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788572306.915652:0:2382:0:(import.c:2195:obd_at_measure()) add 0 to ffff9ada48407c60 time=151 v=5 (5 5 5 5) 00000100:00001000:0.0:1788572306.915656:0:2382:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada48407c18 time=28 v=5 (1 1 1 1) 00000100:00000001:0.0:1788572306.915669:0:2382:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788572306.915672:0:2382:0:(client.c:1506:ptlrpc_check_status()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000001:0.0:1788572306.915674:0:2382:0:(client.c:1712:after_reply()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000040:0.0:1788572306.915680:0:2382:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada455a6d80 x1875449616089984/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/224 e 0 to 0 dl 1788572316 ref 1 fl Rpc:RNQU/200/0 rc -19/-19 job:'ptlrpcd_01_01.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572306.915687:0:2382:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=18446744073709551597 : -19 : 0xffffffffffffffed) 00000100:00000001:0.0:1788572306.915691:0:2382:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788572306.915693:0:2382:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572306.915695:0:2382:0:(import.c:1101:ptlrpc_connect_interpret()) Process entered 00000100:00000001:0.0:1788572306.915697:0:2382:0:(import.c:1136:ptlrpc_connect_interpret()) Process leaving via out (rc=18446744073709551597 : -19 : 0xffffffffffffffed) 00000100:00080000:0.0:1788572306.915702:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada48407800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788572306.915707:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.113@tcp failed (-19) 00000100:00000001:0.0:1788572306.915710:0:2382:0:(import.c:1561:ptlrpc_connect_interpret()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000040:0.0:1788572306.915714:0:2382:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada455a6d80 x1875449616089984/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/224 e 0 to 0 dl 1788572316 ref 1 fl Interpret:RNQU/200/0 rc -19/-19 job:'ptlrpcd_01_01.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572306.915722:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616089984:192.168.202.113@tcp:8:ptlrpcd_01_01.0 00000100:00000001:0.0:1788572306.915727:0:2382:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572306.915730:0:2382:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788572306.915734:0:2382:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455a6d80 x1875449616089984/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/224 e 0 to 0 dl 1788572316 ref 1 fl Complete:RNQU/200/0 rc -19/-19 job:'ptlrpcd_01_01.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572306.915740:0:2382:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788572306.915743:0:2382:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788572306.915747:0:2382:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45652c00. 02000000:00000001:0.0:1788572306.915751:0:2382:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788572306.915755:0:2382:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:0.0:1788572306.915757:0:2382:0:(genops.c:1223:class_import_put()) import ffff9ada48407800 refcount=2 obd=lustre-OST0000-osc-ffff9ada58b0a000 00000020:00000001:0.0:1788572306.915759:0:2382:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:0.0:1788572306.915762:0:2382:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 1024 at ffff9ada607bcc00. 02000000:00000001:0.0:1788572306.915766:0:2382:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788572306.915768:0:2382:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788572306.915770:0:2382:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455a6d80. 00000100:00000001:0.0:1788572306.915775:0:2382:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788572306.915777:0:2382:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572306.915779:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572306.915788:0:2382:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788572306.915790:0:2382:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572306.915792:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788572306.915794:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572307.319002:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572307.319014:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572307.319023:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572307.319024:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572307.319028:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572307.319030:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572307.319033:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572307.319035:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788572307.324350:0:33423:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788572307.324359:0:33423:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788572307.324365:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572308.343527:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572308.343537:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572308.343546:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572308.343547:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572308.343550:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572308.343552:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572308.343555:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572308.343556:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572308.350244:0:15205:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572308.350255:0:15205:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572308.350261:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572309.367005:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572309.367014:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572309.367022:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572309.367023:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572309.367027:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572309.367029:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572309.367033:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572309.367034:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788572309.367127:0:15204:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788572309.367131:0:15204:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788572309.367134:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572310.391034:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572310.391045:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572310.391054:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572310.391055:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572310.391059:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572310.391061:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572310.391065:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572310.391066:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572310.391177:0:33423:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572310.391180:0:33423:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572310.391183:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572311.415655:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572311.415666:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572311.415674:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572311.415676:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572311.415680:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572311.415683:0:365:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:2.0:1788572311.415686:0:365:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:2.0:1788572311.415694:0:365:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:2.0:1788572311.415697:0:365:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:2.0:1788572311.415698:0:365:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572311.415702:0:365:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:2.0:1788572311.415705:0:365:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572311.415707:0:365:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572311.415709:0:365:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572311.415713:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572311.415715:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572311.415884:0:15205:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572311.415888:0:15205:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572311.415891:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:3.0:1788572311.927225:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788572311.927237:0:58077:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788572311.927253:0:58077:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada455a4e00. 00000020:00000040:3.0:1788572311.927265:0:58077:0:(genops.c:1210:class_import_get()) import ffff9ada58a62800 refcount=4 obd=MGC192.168.202.113@tcp 00000100:00000001:3.0:1788572311.927270:0:58077:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788572311.927273:0:58077:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788572311.927281:0:58077:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788572311.927294:0:58077:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada50c31000. 00000100:00000001:3.0:1788572311.927302:0:58077:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572311.927317:0:58077:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS req@ffff9ada455a4e00 x1875449616090112/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788572311.927330:0:58077:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788572311.927335:0:58077:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572311.927342:0:58077:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000005eb88e1a] to pc [ptlrpcd_01_00+0] req@ffff9ada455a4e00 x1875449616090112/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572311.927477:0:58077:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572311.927481:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788572311.927484:0:58077:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788572311.927488:0:58077:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada455a6680. 00000020:00000040:3.0:1788572311.927491:0:58077:0:(genops.c:1210:class_import_get()) import ffff9ada51140000 refcount=4 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000100:00000001:3.0:1788572311.927494:0:58077:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788572311.927496:0:58077:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788572311.927498:0:58077:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788572311.927501:0:58077:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada50c31a00. 00000100:00000001:3.0:1788572311.927504:0:58077:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572311.927510:0:58077:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID req@ffff9ada455a6680 x1875449616090240/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788572311.927518:0:58077:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788572311.927520:0:58077:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572311.927523:0:58077:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d97ec47d] to pc [ptlrpcd_01_01+1] req@ffff9ada455a6680 x1875449616090240/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572311.927531:0:58077:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572311.927534:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788572311.927537:0:58077:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788572311.927540:0:58077:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada455a6d80. 00000020:00000040:3.0:1788572311.927542:0:58077:0:(genops.c:1210:class_import_get()) import ffff9ada58b08000 refcount=4 obd=lustre-MDT0001-mdc-ffff9ada58b0a000 00000100:00000001:3.0:1788572311.927545:0:58077:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788572311.927546:0:58077:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788572311.927548:0:58077:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788572311.927550:0:58077:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada50c31500. 00000100:00000001:3.0:1788572311.927553:0:58077:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572311.927558:0:58077:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID req@ffff9ada455a6d80 x1875449616090368/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788572311.927564:0:58077:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788572311.927565:0:58077:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572311.927569:0:58077:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000009dc80bdd] to pc [ptlrpcd_01_00+0] req@ffff9ada455a6d80 x1875449616090368/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572311.927585:0:58077:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00080000:3.0:1788572311.927588:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572311.927591:0:58077:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00000001:3.0:1788572311.927593:0:58077:0:(import.c:745:ptlrpc_connect_import_locked()) Process entered 00000100:00080000:3.0:1788572311.927596:0:58077:0:(import.c:65:import_set_state_nolock()) ffff9ada48407800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00000001:3.0:1788572311.927600:0:58077:0:(import.c:520:import_select_connection()) Process entered 00000100:00080000:3.0:1788572311.927604:0:58077:0:(import.c:545:import_select_connection()) lustre-OST0000-osc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4224 00000400:00000200:3.0:1788572311.927613:0:58077:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572311.927621:0:58077:0:(peer.c:1689:LNetPeerDiscovered()) Peer NID 192.168.202.113@tcp is discovered: rc = 1 00000100:00001000:3.0:1788572311.927625:0:58077:0:(import.c:2195:obd_at_measure()) add 10 to ffff9ada48407c18 time=33 v=5 (1 1 1 1) 00000100:00001000:3.0:1788572311.927628:0:58077:0:(import.c:2261:obd_at_measure()) AT ffff9ada48407c18 change: old=5 new=10 delta=5 (val=10) hist 10 1 1 1 00000100:00080000:3.0:1788572311.927631:0:58077:0:(import.c:630:import_select_connection()) lustre-OST0000-osc-ffff9ada58b0a000: tried all connections, increasing latency to 5s 00000100:00000040:3.0:1788572311.927635:0:58077:0:(lustre_net.h:1977:ptlrpc_connection_put()) PUT conn=ffff9ada51db1200 refcount 14 to 192.168.202.113@tcp 00000100:00000001:3.0:1788572311.927638:0:58077:0:(connection.c:157:ptlrpc_connection_addref()) Process entered 00000100:00000040:3.0:1788572311.927640:0:58077:0:(connection.c:162:ptlrpc_connection_addref()) conn=ffff9ada51db1200 refcount 15 to 192.168.202.113@tcp 00000100:00000001:3.0:1788572311.927643:0:58077:0:(connection.c:164:ptlrpc_connection_addref()) Process leaving (rc=18446632861199700480 : -111212509851136 : ffff9ada51db1200) 00000020:00000001:3.0:1788572311.927645:0:58077:0:(genops.c:792:class_conn2export()) Process entered 00000020:00000040:3.0:1788572311.927647:0:58077:0:(genops.c:804:class_conn2export()) looking for export cookie 0xef30c340bfb39c83 00000020:00000001:3.0:1788572311.927651:0:58077:0:(lustre_handles.c:130:class_handle2object()) Process entered 00000020:00000040:3.0:1788572311.927655:0:58077:0:(lustre_handles.c:148:class_handle2object()) GET export ffff9ada50c3b800 refcount=4 00000020:00000001:3.0:1788572311.927658:0:58077:0:(lustre_handles.c:155:class_handle2object()) Process leaving (rc=18446632861181392896 : -111212528158720 : ffff9ada50c3b800) 00000020:00000001:3.0:1788572311.927660:0:58077:0:(genops.c:806:class_conn2export()) Process leaving (rc=18446632861181392896 : -111212528158720 : ffff9ada50c3b800) 00000100:00000040:3.0:1788572311.927663:0:58077:0:(lustre_net.h:1977:ptlrpc_connection_put()) PUT conn=ffff9ada51db1200 refcount 14 to 192.168.202.113@tcp 00000100:00000001:3.0:1788572311.927665:0:58077:0:(connection.c:157:ptlrpc_connection_addref()) Process entered 00000100:00000040:3.0:1788572311.927666:0:58077:0:(connection.c:162:ptlrpc_connection_addref()) conn=ffff9ada51db1200 refcount 15 to 192.168.202.113@tcp 00000100:00000001:3.0:1788572311.927668:0:58077:0:(connection.c:164:ptlrpc_connection_addref()) Process leaving (rc=18446632861199700480 : -111212509851136 : ffff9ada51db1200) 00000020:00000040:3.0:1788572311.927671:0:58077:0:(genops.c:879:class_export_put()) PUTting export ffff9ada50c3b800 : new refcount 3 00000100:00080000:3.0:1788572311.927674:0:58077:0:(import.c:665:import_select_connection()) lustre-OST0000-osc-ffff9ada58b0a000: import ffff9ada48407800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00000001:3.0:1788572311.927677:0:58077:0:(import.c:672:import_select_connection()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788572311.927679:0:58077:0:(sec.c:1579:sptlrpc_import_sec_adapt()) Process entered 02000000:00000001:3.0:1788572311.927686:0:58077:0:(sec.c:1619:sptlrpc_import_sec_adapt()) Process leaving via out (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788572311.927689:0:58077:0:(sec.c:1647:sptlrpc_import_sec_adapt()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572311.927691:0:58077:0:(obd_class.h:888:obd_reconnect()) Process entered 00000008:00100000:3.0:1788572311.927696:0:58077:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683394 ocd_grant: 8388608, lost: 0. 00000008:00000001:3.0:1788572311.927699:0:58077:0:(osc_request.c:3833:osc_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572311.927701:0:58077:0:(obd_class.h:902:obd_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788572311.927705:0:58077:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada455a5180. 00000020:00000040:3.0:1788572311.927707:0:58077:0:(genops.c:1210:class_import_get()) import ffff9ada48407800 refcount=3 obd=lustre-OST0000-osc-ffff9ada58b0a000 02000000:00000001:3.0:1788572311.927711:0:58077:0:(sec.c:2038:sptlrpc_sepol_get()) Process entered 02000000:00000001:3.0:1788572311.927713:0:58077:0:(sec.c:2048:sptlrpc_sepol_get()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572311.927715:0:58077:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788572311.927717:0:58077:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788572311.927718:0:58077:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788572311.927722:0:58077:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 1024 at ffff9ada607bf000. 00000100:00000001:3.0:1788572311.927725:0:58077:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788572311.927731:0:58077:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada455a5180 x1875449616090496/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788572311.927738:0:58077:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788572311.927740:0:58077:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572311.927743:0:58077:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000553f1cb3] to pc [ptlrpcd_rcv+-1] req@ffff9ada455a5180 x1875449616090496/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572311.927823:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788572311.927828:0:2382:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788572311.927831:0:2382:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788572311.927838:0:2382:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada455a5180 x1875449616090496/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572311.927845:0:2382:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788572311.927848:0:2382:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572311.927851:0:2382:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788572311.927892:0:2382:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788572311.927895:0:2382:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788572311.927899:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616090496:192.168.202.113@tcp:8:kworker.0 00000100:00000001:0.0:1788572311.927903:0:2382:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788572311.927907:0:2382:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788572311.927909:0:2382:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572311.927912:0:2382:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788572311.927917:0:2382:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at ffff9ada45653000. 02000000:00000001:0.0:1788572311.927919:0:2382:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788572311.927925:0:2382:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:0.0:1788572311.927929:0:2382:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada60acaf20. 00000400:00000010:0.0:1788572311.927937:0:2382:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada5176a088. 00000100:00000200:0.0:1788572311.927947:0:2382:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1875449616090496, portal 4 00000100:00000001:0.0:1788572311.927951:0:2382:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788572311.927952:0:2382:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860989936000 : -111212719615616 : ffff9ada455a5180) 00000100:00000040:0.0:1788572311.927960:0:2382:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada455a5180 x1875449616090496/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572326 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572311.927968:0:2382:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788572311.927971:0:2382:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:0.0:1788572311.927974:0:2382:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada5176a7f8. 00000100:00000200:0.0:1788572311.927978:0:2382:0:(niobuf.c:94:ptl_send_buf()) Sending 520 bytes to portal 28, xid 1875449616090496, offset 0 00000400:00000200:0.0:1788572311.927985:0:2382:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:0.0:1788572311.927994:0:2382:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:0.0:1788572311.928002:0:2382:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:0.0:1788572311.928008:0:2382:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788572311.928012:0:2382:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:0.0:1788572311.928016:0:2382:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788572311.928020:0:2382:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:0.0:1788572311.928024:0:2382:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22274:22274:256:4294967295] 192.168.202.113@tcp LPNI seq info [22270:22270:8:4294967295] 00000400:00000200:0.0:1788572311.928029:0:2382:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:0.0:1788572311.928539:0:2382:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:0.0:1788572311.928548:0:2382:0:(socklnd_cb.c:931:ksocknal_send()) sending 520 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:0.0:1788572311.928556:0:2382:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada5074a000. 00000800:00000200:0.0:1788572311.928563:0:2382:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:0.0:1788572311.928572:0:2382:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:0.0:1788572311.928577:0:2382:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada5074a000 type 1, nob 616 niov 1 nkiov 1 00000100:00000001:0.0:1788572311.928591:0:2382:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572311.928597:0:2382:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: using a timeout of 18862 usecs (19 jiffies) 00000400:00000200:0.0:1788572311.928601:0:2382:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:0.0:1788572311.928605:0:2382:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298896916 00000100:00000001:0.0:1788572311.928608:0:2382:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298902356 00000100:00000001:0.0:1788572311.928610:0:2382:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298896916 00000100:00000001:0.0:1788572311.928612:0:2382:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298902356 00000100:00000001:0.0:1788572311.928615:0:2382:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788572311.928618:0:2382:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572311.928622:0:2382:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572311.928624:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572311.928684:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788572311.928688:0:2385:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788572311.928690:0:2385:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788572311.928698:0:2385:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada455a4e00 x1875449616090112/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572311.928706:0:2385:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788572311.928708:0:2385:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572311.928710:0:2385:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788572311.928713:0:2385:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788572311.928715:0:2385:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788572311.928718:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449616090112:192.168.202.113@tcp:400:kworker.0 00000100:00000001:2.0:1788572311.928722:0:2385:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788572311.928727:0:2385:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788572311.928729:0:2385:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572311.928732:0:2385:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788572311.928737:0:2385:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada45574800. 02000000:00000001:2.0:1788572311.928739:0:2385:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788572311.928743:0:2385:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:2.0:1788572311.928747:0:2385:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada43d96a50. 00000400:00000010:2.0:1788572311.928763:0:2385:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada43c8f110. 00000100:00000200:2.0:1788572311.928768:0:2385:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616090112, portal 25 00000100:00000001:2.0:1788572311.928770:0:2385:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788572311.928771:0:2385:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860989935104 : -111212719616512 : ffff9ada455a4e00) 00000100:00000040:2.0:1788572311.928776:0:2385:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada455a4e00 x1875449616090112/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572327 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572311.928783:0:2385:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788572311.928785:0:2385:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:2.0:1788572311.928788:0:2385:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada43c8f5d8. 00000100:00000200:2.0:1788572311.928790:0:2385:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875449616090112, offset 0 00000400:00000200:2.0:1788572311.928795:0:2385:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:2.0:1788572311.928800:0:2385:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572311.928804:0:2385:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:2.0:1788572311.928808:0:2385:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788572311.928811:0:2385:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:2.0:1788572311.928814:0:2385:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788572311.928816:0:2385:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:2.0:1788572311.928819:0:2385:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22275:22275:255:4294967295] 192.168.202.113@tcp LPNI seq info [22271:22271:7:4294967295] 00000400:00000200:2.0:1788572311.928824:0:2385:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572311.928829:0:2385:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:2.0:1788572311.928833:0:2385:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:2.0:1788572311.928838:0:2385:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada617e8500. 00000800:00000200:2.0:1788572311.928842:0:2385:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:2.0:1788572311.928846:0:2385:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:2.0:1788572311.928848:0:2385:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada617e8500 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788572311.928851:0:2385:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788572311.928886:0:2385:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:2.0:1788572311.928888:0:2385:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS new deadline estimation for cpu 2 is 4298907469 00000100:00000001:2.0:1788572311.928896:0:2385:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4298902356 00000100:00000001:2.0:1788572311.928902:0:2385:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS new deadline estimation for cpu 3 is 4298907469 00000100:00000001:2.0:1788572311.928905:0:2385:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4298902356 00000100:00000001:2.0:1788572311.928908:0:2385:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788572311.928910:0:2385:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572311.928913:0:2385:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788572311.928917:0:2385:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada455a6d80 x1875449616090368/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572311.928924:0:2385:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788572311.928926:0:2385:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572311.928928:0:2385:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788572311.928930:0:2385:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788572311.928932:0:2385:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788572311.928935:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616090368:192.168.202.113@tcp:400:kworker.0 00000100:00000001:2.0:1788572311.928939:0:2385:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788572311.928943:0:2385:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788572311.928944:0:2385:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572311.928946:0:2385:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788572311.928950:0:2385:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada45575200. 02000000:00000001:2.0:1788572311.928952:0:2385:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788572311.928956:0:2385:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:2.0:1788572311.928959:0:2385:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada43d96cb8. 00000400:00000010:2.0:1788572311.928963:0:2385:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada43c8faa0. 00000100:00000200:2.0:1788572311.928966:0:2385:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616090368, portal 10 00000100:00000001:2.0:1788572311.928968:0:2385:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788572311.928969:0:2385:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860989943168 : -111212719608448 : ffff9ada455a6d80) 00000100:00000040:2.0:1788572311.928976:0:2385:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada455a6d80 x1875449616090368/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572327 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572311.928983:0:2385:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788572311.928985:0:2385:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:2.0:1788572311.928988:0:2385:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada43c8f990. 00000100:00000200:2.0:1788572311.928990:0:2385:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875449616090368, offset 0 00000400:00000200:2.0:1788572311.928995:0:2385:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:2.0:1788572311.928998:0:2385:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572311.929001:0:2385:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000100:00000001:3.0:1788572311.929196:0:58077:0:(import.c:877:ptlrpc_connect_import_locked()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572311.929201:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:3.0:1788572311.929206:0:58077:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:3.0:1788572311.929212:0:58077:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (4234) 00000100:00000001:3.0:1788572311.929231:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788572311.929234:0:2386:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788572311.929236:0:2386:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788572311.929244:0:2386:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada455a6680 x1875449616090240/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572311.929251:0:2386:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788572311.929253:0:2386:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788572311.929256:0:2386:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788572311.929257:0:2386:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788572311.929259:0:2386:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788572311.929262:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616090240:192.168.202.113@tcp:400:kworker.0 00000100:00000001:3.0:1788572311.929266:0:2386:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788572311.929269:0:2386:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788572311.929270:0:2386:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788572311.929272:0:2386:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788572311.929276:0:2386:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada50a3be00. 02000000:00000001:3.0:1788572311.929278:0:2386:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788572311.929282:0:2386:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:3.0:1788572311.929285:0:2386:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada510c92c0. 00000400:00000010:3.0:1788572311.929289:0:2386:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada51269b28. 00000100:00000200:3.0:1788572311.929292:0:2386:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616090240, portal 10 00000100:00000001:3.0:1788572311.929294:0:2386:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788572311.929295:0:2386:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860989941376 : -111212719610240 : ffff9ada455a6680) 00000100:00000040:3.0:1788572311.929301:0:2386:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada455a6680 x1875449616090240/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572327 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572311.929305:0:2386:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788572311.929308:0:2386:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:3.0:1788572311.929310:0:2386:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada51269550. 00000100:00000200:3.0:1788572311.929311:0:2386:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875449616090240, offset 0 00000400:00000200:3.0:1788572311.929314:0:2386:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:2.0:1788572311.929329:0:2385:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788572311.929333:0:2385:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:2.0:1788572311.929334:0:2385:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788572311.929336:0:2385:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:2.0:1788572311.929338:0:2385:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22276:22276:254:4294967295] 192.168.202.113@tcp LPNI seq info [22272:22272:6:4294967295] 00000400:00000200:2.0:1788572311.929341:0:2385:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572311.929347:0:2385:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:2.0:1788572311.929429:0:2385:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:2.0:1788572311.929434:0:2385:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada617e8f00. 00000800:00000200:2.0:1788572311.929439:0:2385:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:2.0:1788572311.929444:0:2385:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:2.0:1788572311.929447:0:2385:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada617e8f00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788572311.929451:0:2385:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572311.929454:0:2385:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: using a timeout of 16550 usecs (17 jiffies) 00000400:00000200:2.0:1788572311.929457:0:2385:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:2.0:1788572311.929460:0:2385:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298896915 00000100:00000001:2.0:1788572311.929462:0:2385:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298907469 00000100:00000001:2.0:1788572311.929464:0:2385:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298896915 00000100:00000001:2.0:1788572311.929466:0:2385:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298907469 00000100:00000001:2.0:1788572311.929469:0:2385:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788572311.929471:0:2385:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572311.929473:0:2385:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572311.929476:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572311.929484:0:2385:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788572311.929485:0:2385:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788572311.929487:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788572311.929488:0:2385:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788572311.929490:0:2385:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572311.929492:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788572311.929535:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788572311.929539:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada5074a000. 00000400:00000200:2.0:1788572311.929546:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572311.929550:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:0.0:1788572311.930532:0:2382:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788572311.930536:0:2382:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=5 : 5 : 5) 00000100:00000001:0.0:1788572311.930540:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788572311.930542:0:2382:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788572311.930545:0:2382:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572311.930547:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788572311.932397:0:2386:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572311.932417:0:2386:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:3.0:1788572311.932422:0:2386:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788572311.932428:0:2386:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:3.0:1788572311.932432:0:2386:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788572311.932435:0:2386:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:3.0:1788572311.932440:0:2386:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22277:22277:253:4294967295] 192.168.202.113@tcp LPNI seq info [22273:22273:5:4294967295] 00000400:00000200:3.0:1788572311.932446:0:2386:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572311.932457:0:2386:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:3.0:1788572311.932465:0:2386:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:3.0:1788572311.932480:0:2386:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada50c31300. 00000800:00000200:3.0:1788572311.932488:0:2386:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:3.0:1788572311.932496:0:2386:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:3.0:1788572311.932501:0:2386:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada50c31300 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788572311.932508:0:2386:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572311.932515:0:2386:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: using a timeout of 41272 usecs (42 jiffies) 00000400:00000200:3.0:1788572311.932520:0:2386:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:3.0:1788572311.932523:0:2386:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298896943 00000100:00000001:3.0:1788572311.932528:0:2386:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298907469 00000100:00000001:3.0:1788572311.932530:0:2386:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298896943 00000100:00000001:3.0:1788572311.932533:0:2386:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298907469 00000100:00000001:3.0:1788572311.932536:0:2386:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788572311.932540:0:2386:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572311.932544:0:2386:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572311.932548:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788572311.932559:0:2386:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788572311.932563:0:2386:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788572311.932566:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788572311.932568:0:2386:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788572311.932570:0:2386:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572311.932573:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788572311.935810:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada5176a7f8 00000400:00000010:2.0:1788572311.935823:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada5176a7f8. 00000100:00000001:2.0:1788572311.935836:0:2377:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:2.0:1788572311.935851:0:2377:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada455a5180 x1875449616090496/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572326 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572311.936258:0:2377:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572311.936262:0:2377:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada455a5180 x1875449616090496/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572326 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572311.936268:0:2377:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572311.936270:0:2377:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000200:2.0:1788572311.936314:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788572311.936318:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada617e8500. 00000400:00000200:2.0:1788572311.936323:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572311.936328:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:2.0:1788572311.936331:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada43c8f5d8 00000400:00000010:2.0:1788572311.936332:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada43c8f5d8. 00000100:00000001:2.0:1788572311.936335:0:2377:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:2.0:1788572311.936338:0:2377:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada455a4e00 x1875449616090112/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572327 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572311.936345:0:2377:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572311.936346:0:2377:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada455a4e00 x1875449616090112/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572327 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572311.936350:0:2377:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572311.936351:0:2377:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000200:2.0:1788572311.936356:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788572311.936357:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada617e8f00. 00000400:00000200:2.0:1788572311.936359:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572311.936361:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:2.0:1788572311.936363:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada43c8f990 00000400:00000010:2.0:1788572311.936364:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada43c8f990. 00000100:00000001:2.0:1788572311.936366:0:2377:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:2.0:1788572311.936368:0:2377:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada455a6d80 x1875449616090368/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572327 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572311.936373:0:2377:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572311.936374:0:2377:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada455a6d80 x1875449616090368/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572327 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572311.936378:0:2377:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572311.936378:0:2377:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000200:2.0:1788572311.936542:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788572311.936545:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada50c31300. 00000400:00000200:2.0:1788572311.936549:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572311.936552:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:2.0:1788572311.936555:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada51269550 00000400:00000010:2.0:1788572311.936557:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada51269550. 00000100:00000001:2.0:1788572311.936560:0:2377:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:2.0:1788572311.936564:0:2377:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada455a6680 x1875449616090240/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572327 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572311.936572:0:2377:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572311.936621:0:2377:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada455a6680 x1875449616090240/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572327 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572311.936627:0:2377:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572311.936628:0:2377:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000001:3.0:1788572311.940891:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572311.941071:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572311.941077:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572311.941081:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572311.941092:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572311.941103:0:2378:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572311.941114:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572311.941120:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572311.941126:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572311.941132:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 25 MB=0x6a9b624294400 00000400:00000200:3.0:1788572311.941137:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572311.941143:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.202.113@tcp of length 224/224 into md 0x356b5 [1] + 224 00000800:00000001:3.0:1788572311.941153:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572311.941157:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572311.941161:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572311.941166:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572311.941187:0:2378:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572311.941199:0:2378:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada455a4e00 x1875449616090112/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572327 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572311.941213:0:2378:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada455a4e00 x1875449616090112/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572327 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572311.941236:0:2378:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572311.941246:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572311.941250:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572311.941252:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572311.941254:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572311.941257:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572311.941260:0:2378:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572311.941264:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572311.941268:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572311.941270:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572311.941273:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 10 MB=0x6a9b624294500 00000400:00000200:3.0:1788572311.941276:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572311.941280:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.113@tcp of length 224/224 into md 0x356c5 [1] + 224 00000800:00000001:3.0:1788572311.941284:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572311.941286:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572311.941289:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572311.941291:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572311.941294:0:2378:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572311.941299:0:2378:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada455a6d80 x1875449616090368/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572327 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572311.941307:0:2378:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada455a6d80 x1875449616090368/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572327 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572311.941314:0:2378:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572311.941317:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572311.941320:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572311.941321:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572311.941323:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572311.941325:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572311.941329:0:2378:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572311.941332:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572311.941335:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572311.941338:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572311.941340:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 10 MB=0x6a9b624294480 00000400:00000200:3.0:1788572311.941343:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572311.941347:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.113@tcp of length 224/224 into md 0x356d5 [1] + 224 00000800:00000001:3.0:1788572311.941351:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572311.941356:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572311.941358:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572311.941361:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572311.941364:0:2378:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572311.941369:0:2378:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada455a6680 x1875449616090240/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572327 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572311.941377:0:2378:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada455a6680 x1875449616090240/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572327 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572311.941393:0:2378:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572311.941397:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572311.941400:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572311.941407:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788572311.941410:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788572311.941412:0:2385:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788572311.941412:0:2386:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788572311.941420:0:2386:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788572311.941423:0:2386:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788572311.941488:0:2386:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada510c92c0. 00000400:00000200:3.0:1788572311.941494:0:2386:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada51269b28 00000400:00000010:3.0:1788572311.941498:0:2386:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada51269b28. 00000100:00000001:3.0:1788572311.941501:0:2386:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572311.941506:0:2386:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada455a6680 x1875449616090240/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572327 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788572311.941514:0:2386:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada455a6680 x1875449616090240/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572327 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572311.941520:0:2386:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788572311.941523:0:2386:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788572311.941525:0:2386:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:3.0:1788572311.941530:0:2386:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788572311.941533:0:2386:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788572311.941535:0:2386:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788572311.941538:0:2386:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788572311.941551:0:2386:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada51140460 time=6 v=5 (5 5 5 5) 00000100:00001000:3.0:1788572311.941556:0:2386:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada51140418 time=102 v=5 (2 0 0 0) 00000100:00000001:3.0:1788572311.941559:0:2386:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788572311.941561:0:2386:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788572311.941564:0:2386:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788572311.941569:0:2386:0:(ldlm_request.c:1703:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572311.941574:0:2386:0:(client.c:3110:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788572311.941576:0:2386:0:(client.c:3122:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: skip recheck: last_committed 219043334508 00000100:00000001:3.0:1788572311.941578:0:2386:0:(client.c:3123:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788572311.941580:0:2386:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572311.941585:0:2386:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada455a6680 x1875449616090240/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572327 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572311.941592:0:2386:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788572311.941595:0:2386:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788572311.941598:0:2386:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788572311.941602:0:2386:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada455a6680 x1875449616090240/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572327 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572311.941610:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616090240:192.168.202.113@tcp:400:kworker.0 00000100:00000001:3.0:1788572311.941615:0:2386:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788572311.941618:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572311.941622:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455a6680 x1875449616090240/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572327 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572311.941628:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788572311.941631:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788572311.941635:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada50a3be00. 00000400:00000001:2.0:1788572311.941636:0:2385:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788572311.941638:0:2385:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 02000000:00000001:3.0:1788572311.941639:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000400:00000010:2.0:1788572311.941640:0:2385:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada43d96a50. 00000020:00000001:3.0:1788572311.941642:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000400:00000200:2.0:1788572311.941644:0:2385:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada43c8f110 00000020:00000040:3.0:1788572311.941644:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=3 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000400:00000010:2.0:1788572311.941646:0:2385:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada43c8f110. 00000100:00000001:2.0:1788572311.941649:0:2385:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:2.0:1788572311.941655:0:2385:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada455a4e00 x1875449616090112/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572327 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000020:00000001:3.0:1788572311.941656:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:3.0:1788572311.941659:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada50c31a00. 00000100:00000200:2.0:1788572311.941663:0:2385:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada455a4e00 x1875449616090112/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572327 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1788572311.941664:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788572311.941666:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788572311.941668:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455a6680. 00000100:00000001:2.0:1788572311.941669:0:2385:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788572311.941671:0:2385:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788572311.941672:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572311.941673:0:2385:0:(client.c:1593:after_reply()) Process entered 00000100:00000001:3.0:1788572311.941673:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:2.0:1788572311.941674:0:2385:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788572311.941676:0:2385:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788572311.941676:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572311.941677:0:2385:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572311.941679:0:2385:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788572311.941683:0:2385:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada58a62c60 time=73 v=5 (5 5 5 5) 00000100:00000001:3.0:1788572311.941684:0:2386:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00001000:2.0:1788572311.941686:0:2385:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada58a62c18 time=103 v=5 (1 0 0 0) 00000100:00000001:3.0:1788572311.941686:0:2386:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572311.941688:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788572311.941689:0:2385:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788572311.941690:0:2385:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572311.941691:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572311.941692:0:2385:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788572311.941693:0:2385:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572311.941695:0:2385:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572311.941698:0:2385:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada455a4e00 x1875449616090112/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572327 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572311.941705:0:2385:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788572311.941706:0:2385:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788572311.941708:0:2385:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572311.941711:0:2385:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada455a4e00 x1875449616090112/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572327 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788572311.941718:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455a4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449616090112:192.168.202.113@tcp:400:kworker.0 00000400:00000001:2.0:1788572311.941723:0:2385:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788572311.941724:0:2385:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788572311.941726:0:2385:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada43d96cb8. 00000400:00000200:2.0:1788572311.941728:0:2385:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada43c8faa0 00000400:00000010:2.0:1788572311.941730:0:2385:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada43c8faa0. 00000100:00000001:2.0:1788572311.941732:0:2385:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:2.0:1788572311.941736:0:2385:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada455a6d80 x1875449616090368/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572327 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788572311.941743:0:2385:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada455a6d80 x1875449616090368/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572327 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572311.941749:0:2385:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788572311.941750:0:2385:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572311.941752:0:2385:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:2.0:1788572311.941753:0:2385:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788572311.941754:0:2385:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788572311.941755:0:2385:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572311.941757:0:2385:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788572311.941762:0:2385:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada58b08460 time=73 v=5 (5 5 5 5) 00000100:00001000:2.0:1788572311.941765:0:2385:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada58b08418 time=73 v=5 (1 1 1 1) 00000100:00000001:2.0:1788572311.941767:0:2385:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788572311.941769:0:2385:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572311.941771:0:2385:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788572311.941774:0:2385:0:(ldlm_request.c:1703:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572311.941777:0:2385:0:(client.c:3110:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788572311.941778:0:2385:0:(client.c:3122:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff9ada58b0a000: skip recheck: last_committed 0 00000100:00000001:2.0:1788572311.941780:0:2385:0:(client.c:3123:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788572311.941781:0:2385:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572311.941785:0:2385:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada455a6d80 x1875449616090368/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572327 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572311.941790:0:2385:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788572311.941792:0:2385:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788572311.941793:0:2385:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572311.941796:0:2385:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada455a6d80 x1875449616090368/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572327 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788572311.941803:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616090368:192.168.202.113@tcp:400:kworker.0 00000100:00000001:2.0:1788572311.941807:0:2385:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572311.941820:0:2385:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572311.941823:0:2385:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455a4e00 x1875449616090112/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572327 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572311.941829:0:2385:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572311.941830:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572311.941832:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada45574800. 02000000:00000001:2.0:1788572311.941835:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572311.941837:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572311.941838:0:2385:0:(genops.c:1223:class_import_put()) import ffff9ada58a62800 refcount=3 obd=MGC192.168.202.113@tcp 00000020:00000001:2.0:1788572311.941840:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572311.941841:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada50c31000. 02000000:00000001:2.0:1788572311.941844:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572311.941845:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572311.941846:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455a4e00. 00000100:00000001:2.0:1788572311.941850:0:2385:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572311.942056:0:2385:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572311.942060:0:2385:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572311.942064:0:2385:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455a6d80 x1875449616090368/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572327 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572311.942209:0:2385:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572311.942210:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572311.942213:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada45575200. 02000000:00000001:2.0:1788572311.942216:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572311.942217:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572311.942219:0:2385:0:(genops.c:1223:class_import_put()) import ffff9ada58b08000 refcount=3 obd=lustre-MDT0001-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572311.942221:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572311.942222:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada50c31500. 02000000:00000001:2.0:1788572311.942225:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572311.942226:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572311.942227:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455a6d80. 00000100:00000001:2.0:1788572311.942230:0:2385:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572311.942231:0:2385:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572311.942233:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572311.942238:0:2385:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788572311.942240:0:2385:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572311.942241:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788572311.942243:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788572311.943400:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572311.943417:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788572311.943422:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572311.943425:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572311.943446:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572311.943454:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:2.0:1788572311.943463:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572311.943468:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572311.943473:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572311.943478:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 4 MB=0x6a9b624294580 00000400:00000200:2.0:1788572311.943484:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572311.943489:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.202.113@tcp of length 224/224 into md 0x356ad [1] + 0 00000800:00000001:2.0:1788572311.943497:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572311.943523:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572311.943527:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572311.943533:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:2.0:1788572311.943542:0:2377:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:2.0:1788572311.943550:0:2377:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada455a5180 x1875449616090496/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572326 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788572311.943568:0:2377:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=0 replen=544 req@ffff9ada455a5180 x1875449616090496/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572326 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572311.943601:0:2377:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788572311.943611:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572311.943615:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572311.943778:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788572311.943782:0:2382:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788572311.943789:0:2382:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788572311.943792:0:2382:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788572311.943795:0:2382:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada60acaf20. 00000400:00000200:0.0:1788572311.943800:0:2382:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada5176a088 00000400:00000010:0.0:1788572311.943813:0:2382:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada5176a088. 00000100:00000001:0.0:1788572311.943817:0:2382:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:0.0:1788572311.943821:0:2382:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada455a5180 x1875449616090496/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572326 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788572311.943829:0:2382:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada455a5180 x1875449616090496/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572326 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572311.943836:0:2382:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788572311.943837:0:2382:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572311.943840:0:2382:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:0.0:1788572311.943843:0:2382:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788572311.943844:0:2382:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788572311.943846:0:2382:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572311.943848:0:2382:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788572311.944134:0:2382:0:(import.c:2195:obd_at_measure()) add 0 to ffff9ada48407c60 time=156 v=5 (5 5 5 5) 00000100:00001000:0.0:1788572311.944139:0:2382:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada48407c18 time=33 v=10 (10 1 1 1) 00000100:00000001:0.0:1788572311.944142:0:2382:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788572311.944145:0:2382:0:(client.c:1506:ptlrpc_check_status()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000001:0.0:1788572311.944147:0:2382:0:(client.c:1712:after_reply()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000040:0.0:1788572311.944153:0:2382:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada455a5180 x1875449616090496/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/224 e 0 to 0 dl 1788572326 ref 1 fl Rpc:RNQU/200/0 rc -19/-19 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572311.944161:0:2382:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=18446744073709551597 : -19 : 0xffffffffffffffed) 00000100:00000001:0.0:1788572311.944164:0:2382:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788572311.944166:0:2382:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572311.944168:0:2382:0:(import.c:1101:ptlrpc_connect_interpret()) Process entered 00000100:00000001:0.0:1788572311.944170:0:2382:0:(import.c:1136:ptlrpc_connect_interpret()) Process leaving via out (rc=18446744073709551597 : -19 : 0xffffffffffffffed) 00000100:00080000:0.0:1788572311.944174:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada48407800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788572311.944190:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.113@tcp failed (-19) 00000100:00000001:0.0:1788572311.944193:0:2382:0:(import.c:1561:ptlrpc_connect_interpret()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000040:0.0:1788572311.944198:0:2382:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada455a5180 x1875449616090496/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/224 e 0 to 0 dl 1788572326 ref 1 fl Interpret:RNQU/200/0 rc -19/-19 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572311.944205:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455a5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616090496:192.168.202.113@tcp:8:kworker.0 00000100:00000001:0.0:1788572311.944210:0:2382:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572311.944212:0:2382:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788572311.944215:0:2382:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455a5180 x1875449616090496/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/224 e 0 to 0 dl 1788572326 ref 1 fl Complete:RNQU/200/0 rc -19/-19 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572311.944222:0:2382:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788572311.944224:0:2382:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788572311.944228:0:2382:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45653000. 02000000:00000001:0.0:1788572311.944234:0:2382:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788572311.944235:0:2382:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:0.0:1788572311.944237:0:2382:0:(genops.c:1223:class_import_put()) import ffff9ada48407800 refcount=2 obd=lustre-OST0000-osc-ffff9ada58b0a000 00000020:00000001:0.0:1788572311.944240:0:2382:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:0.0:1788572311.944242:0:2382:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 1024 at ffff9ada607bf000. 02000000:00000001:0.0:1788572311.944245:0:2382:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788572311.944247:0:2382:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788572311.944248:0:2382:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455a5180. 00000100:00000001:0.0:1788572311.944256:0:2382:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788572311.944257:0:2382:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572311.944259:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572311.944267:0:2382:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788572311.944269:0:2382:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572311.944271:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788572311.944272:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572312.440782:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572312.440793:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572312.440801:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572312.440802:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572312.440806:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572312.440816:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572312.440821:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572312.440824:0:365:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:2.0:1788572312.440827:0:365:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:2.0:1788572312.440835:0:365:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:2.0:1788572312.440838:0:365:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:2.0:1788572312.440840:0:365:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572312.440844:0:365:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:2.0:1788572312.440846:0:365:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572312.440848:0:365:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572312.440851:0:365:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788572312.441551:0:15204:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788572312.441557:0:15204:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788572312.441561:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572313.464626:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572313.464640:0:365:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:2.0:1788572313.464642:0:365:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:2.0:1788572313.464647:0:365:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572313.464654:0:365:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:2.0:1788572313.464657:0:365:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572313.464660:0:365:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572313.464663:0:365:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572313.464667:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572313.464669:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572313.464673:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572313.464674:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572313.464678:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572313.464680:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572313.464849:0:33423:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572313.464873:0:33423:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572313.464877:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572314.491495:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572314.491505:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572314.491514:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572314.491515:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572314.491519:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572314.491520:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572314.491524:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572314.491525:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572314.492063:0:15205:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572314.492068:0:15205:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572314.492071:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572315.511018:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572315.511026:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572315.511032:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572315.511037:0:365:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:2.0:1788572315.511039:0:365:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:2.0:1788572315.511043:0:365:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572315.511046:0:365:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:2.0:1788572315.511048:0:365:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572315.511051:0:365:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572315.511053:0:365:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572315.511057:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572315.511058:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572315.511062:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572315.511063:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788572315.511176:0:15204:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788572315.511180:0:15204:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788572315.511183:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572316.534979:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572316.534988:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572316.534995:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572316.534997:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572316.535001:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572316.535002:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572316.535006:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572316.535007:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572316.538937:0:33423:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572316.538948:0:33423:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572316.538955:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:3.0:1788572317.047895:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788572317.047910:0:77466:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788572317.047925:0:77466:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada455a6680. 00000020:00000040:3.0:1788572317.047933:0:77466:0:(genops.c:1210:class_import_get()) import ffff9ada58a62800 refcount=4 obd=MGC192.168.202.113@tcp 00000100:00000001:3.0:1788572317.047947:0:77466:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788572317.047950:0:77466:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788572317.047956:0:77466:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788572317.047969:0:77466:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada50c31a00. 00000100:00000001:3.0:1788572317.047977:0:77466:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572317.047994:0:77466:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS req@ffff9ada455a6680 x1875449616090624/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788572317.048005:0:77466:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788572317.048012:0:77466:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572317.048018:0:77466:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d97ec47d] to pc [ptlrpcd_01_01+1] req@ffff9ada455a6680 x1875449616090624/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572317.048056:0:77466:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572317.048059:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788572317.048063:0:77466:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788572317.048066:0:77466:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada455a4380. 00000020:00000040:3.0:1788572317.048069:0:77466:0:(genops.c:1210:class_import_get()) import ffff9ada51140000 refcount=4 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000100:00000001:3.0:1788572317.048071:0:77466:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788572317.048072:0:77466:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788572317.048074:0:77466:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788572317.048077:0:77466:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada50c31600. 00000100:00000001:3.0:1788572317.048080:0:77466:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572317.048085:0:77466:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID req@ffff9ada455a4380 x1875449616090752/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788572317.048092:0:77466:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788572317.048093:0:77466:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572317.048097:0:77466:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000005880b1af] to pc [ptlrpcd_01_00+0] req@ffff9ada455a4380 x1875449616090752/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572317.048105:0:77466:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572317.048108:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788572317.048111:0:77466:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788572317.048113:0:77466:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada455a5500. 00000020:00000040:3.0:1788572317.048115:0:77466:0:(genops.c:1210:class_import_get()) import ffff9ada58b08000 refcount=4 obd=lustre-MDT0001-mdc-ffff9ada58b0a000 00000100:00000001:3.0:1788572317.048117:0:77466:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788572317.048119:0:77466:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788572317.048120:0:77466:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788572317.048123:0:77466:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada50c31400. 00000100:00000001:3.0:1788572317.048126:0:77466:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572317.048129:0:77466:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID req@ffff9ada455a5500 x1875449616090880/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788572317.048136:0:77466:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788572317.048138:0:77466:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572317.048142:0:77466:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000004bcb444d] to pc [ptlrpcd_01_01+1] req@ffff9ada455a5500 x1875449616090880/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572317.048148:0:77466:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00080000:3.0:1788572317.048150:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572317.048154:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00000001:3.0:1788572317.048157:0:77466:0:(import.c:745:ptlrpc_connect_import_locked()) Process entered 00000100:00080000:3.0:1788572317.048160:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada48407800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00000001:3.0:1788572317.048164:0:77466:0:(import.c:520:import_select_connection()) Process entered 00000100:00080000:3.0:1788572317.048167:0:77466:0:(import.c:545:import_select_connection()) lustre-OST0000-osc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4229 00000400:00000200:3.0:1788572317.048177:0:77466:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572317.048184:0:77466:0:(peer.c:1689:LNetPeerDiscovered()) Peer NID 192.168.202.113@tcp is discovered: rc = 1 00000100:00001000:3.0:1788572317.048188:0:77466:0:(import.c:2195:obd_at_measure()) add 15 to ffff9ada48407c18 time=39 v=10 (10 1 1 1) 00000100:00001000:3.0:1788572317.048192:0:77466:0:(import.c:2261:obd_at_measure()) AT ffff9ada48407c18 change: old=10 new=15 delta=5 (val=15) hist 15 1 1 1 00000100:00080000:3.0:1788572317.048195:0:77466:0:(import.c:630:import_select_connection()) lustre-OST0000-osc-ffff9ada58b0a000: tried all connections, increasing latency to 10s 00000100:00000040:3.0:1788572317.048199:0:77466:0:(lustre_net.h:1977:ptlrpc_connection_put()) PUT conn=ffff9ada51db1200 refcount 14 to 192.168.202.113@tcp 00000100:00000001:3.0:1788572317.048202:0:77466:0:(connection.c:157:ptlrpc_connection_addref()) Process entered 00000100:00000040:3.0:1788572317.048205:0:77466:0:(connection.c:162:ptlrpc_connection_addref()) conn=ffff9ada51db1200 refcount 15 to 192.168.202.113@tcp 00000100:00000001:3.0:1788572317.048206:0:77466:0:(connection.c:164:ptlrpc_connection_addref()) Process leaving (rc=18446632861199700480 : -111212509851136 : ffff9ada51db1200) 00000020:00000001:3.0:1788572317.048209:0:77466:0:(genops.c:792:class_conn2export()) Process entered 00000020:00000040:3.0:1788572317.048211:0:77466:0:(genops.c:804:class_conn2export()) looking for export cookie 0xef30c340bfb39c83 00000020:00000001:3.0:1788572317.048214:0:77466:0:(lustre_handles.c:130:class_handle2object()) Process entered 00000020:00000040:3.0:1788572317.048218:0:77466:0:(lustre_handles.c:148:class_handle2object()) GET export ffff9ada50c3b800 refcount=4 00000020:00000001:3.0:1788572317.048220:0:77466:0:(lustre_handles.c:155:class_handle2object()) Process leaving (rc=18446632861181392896 : -111212528158720 : ffff9ada50c3b800) 00000020:00000001:3.0:1788572317.048222:0:77466:0:(genops.c:806:class_conn2export()) Process leaving (rc=18446632861181392896 : -111212528158720 : ffff9ada50c3b800) 00000100:00000040:3.0:1788572317.048225:0:77466:0:(lustre_net.h:1977:ptlrpc_connection_put()) PUT conn=ffff9ada51db1200 refcount 14 to 192.168.202.113@tcp 00000100:00000001:3.0:1788572317.048226:0:77466:0:(connection.c:157:ptlrpc_connection_addref()) Process entered 00000100:00000040:3.0:1788572317.048228:0:77466:0:(connection.c:162:ptlrpc_connection_addref()) conn=ffff9ada51db1200 refcount 15 to 192.168.202.113@tcp 00000100:00000001:3.0:1788572317.048229:0:77466:0:(connection.c:164:ptlrpc_connection_addref()) Process leaving (rc=18446632861199700480 : -111212509851136 : ffff9ada51db1200) 00000020:00000040:3.0:1788572317.048232:0:77466:0:(genops.c:879:class_export_put()) PUTting export ffff9ada50c3b800 : new refcount 3 00000100:00080000:3.0:1788572317.048236:0:77466:0:(import.c:665:import_select_connection()) lustre-OST0000-osc-ffff9ada58b0a000: import ffff9ada48407800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00000001:3.0:1788572317.048238:0:77466:0:(import.c:672:import_select_connection()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788572317.048241:0:77466:0:(sec.c:1579:sptlrpc_import_sec_adapt()) Process entered 02000000:00000001:3.0:1788572317.048248:0:77466:0:(sec.c:1619:sptlrpc_import_sec_adapt()) Process leaving via out (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788572317.048262:0:77466:0:(sec.c:1647:sptlrpc_import_sec_adapt()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572317.048265:0:77466:0:(obd_class.h:888:obd_reconnect()) Process entered 00000008:00100000:3.0:1788572317.048271:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683394 ocd_grant: 8388608, lost: 0. 00000008:00000001:3.0:1788572317.048273:0:77466:0:(osc_request.c:3833:osc_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572317.048275:0:77466:0:(obd_class.h:902:obd_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:3.0:1788572317.048280:0:77466:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada455a7100. 00000020:00000040:3.0:1788572317.048282:0:77466:0:(genops.c:1210:class_import_get()) import ffff9ada48407800 refcount=3 obd=lustre-OST0000-osc-ffff9ada58b0a000 02000000:00000001:3.0:1788572317.048286:0:77466:0:(sec.c:2038:sptlrpc_sepol_get()) Process entered 02000000:00000001:3.0:1788572317.048287:0:77466:0:(sec.c:2048:sptlrpc_sepol_get()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572317.048289:0:77466:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788572317.048291:0:77466:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788572317.048293:0:77466:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788572317.048297:0:77466:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 1024 at ffff9ada607bd000. 00000100:00000001:3.0:1788572317.048300:0:77466:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788572317.048306:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada455a7100 x1875449616091008/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788572317.048314:0:77466:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788572317.048326:0:77466:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572317.048330:0:77466:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000f47f7eea] to pc [ptlrpcd_rcv+-1] req@ffff9ada455a7100 x1875449616091008/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572317.048350:0:77466:0:(import.c:877:ptlrpc_connect_import_locked()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572317.048354:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:3.0:1788572317.048358:0:77466:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:3.0:1788572317.048364:0:77466:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (4239) 00000100:00000001:0.0:1788572317.049133:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788572317.049139:0:2382:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788572317.049142:0:2382:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788572317.049152:0:2382:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada455a7100 x1875449616091008/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572317.049163:0:2382:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788572317.049165:0:2382:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572317.049169:0:2382:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788572317.049172:0:2382:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788572317.049175:0:2382:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788572317.049178:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616091008:192.168.202.113@tcp:8:kworker.0 00000100:00000001:0.0:1788572317.049183:0:2382:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788572317.049189:0:2382:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788572317.049191:0:2382:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572317.049194:0:2382:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788572317.049201:0:2382:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at ffff9ada45653000. 02000000:00000001:0.0:1788572317.049203:0:2382:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788572317.049210:0:2382:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:0.0:1788572317.049214:0:2382:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada60acaf20. 00000400:00000010:0.0:1788572317.049222:0:2382:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada5176a088. 00000100:00000200:0.0:1788572317.049232:0:2382:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1875449616091008, portal 4 00000100:00000001:0.0:1788572317.049235:0:2382:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788572317.049237:0:2382:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860989944064 : -111212719607552 : ffff9ada455a7100) 00000100:00000040:0.0:1788572317.049245:0:2382:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada455a7100 x1875449616091008/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572337 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572317.049252:0:2382:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788572317.049256:0:2382:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:0.0:1788572317.049271:0:2382:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada5176a330. 00000100:00000200:0.0:1788572317.049275:0:2382:0:(niobuf.c:94:ptl_send_buf()) Sending 520 bytes to portal 28, xid 1875449616091008, offset 0 00000400:00000200:0.0:1788572317.049282:0:2382:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:0.0:1788572317.049293:0:2382:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:0.0:1788572317.049301:0:2382:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:0.0:1788572317.049307:0:2382:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788572317.049312:0:2382:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:0.0:1788572317.049326:0:2382:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788572317.049329:0:2382:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:0.0:1788572317.049333:0:2382:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22278:22278:256:4294967295] 192.168.202.113@tcp LPNI seq info [22274:22274:8:4294967295] 00000400:00000200:0.0:1788572317.049338:0:2382:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:0.0:1788572317.049348:0:2382:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:0.0:1788572317.049356:0:2382:0:(socklnd_cb.c:931:ksocknal_send()) sending 520 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:0.0:1788572317.049361:0:2382:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada5074ad00. 00000800:00000200:0.0:1788572317.049368:0:2382:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:0.0:1788572317.049377:0:2382:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:0.0:1788572317.049382:0:2382:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada5074ad00 type 1, nob 616 niov 1 nkiov 1 00000100:00000001:0.0:1788572317.049399:0:2382:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572317.049413:0:2382:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: using a timeout of 18863 usecs (19 jiffies) 00000400:00000200:0.0:1788572317.049416:0:2382:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:0.0:1788572317.049419:0:2382:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298902037 00000100:00000001:0.0:1788572317.049422:0:2382:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298907469 00000100:00000001:0.0:1788572317.049424:0:2382:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298902037 00000100:00000001:0.0:1788572317.049426:0:2382:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298907469 00000100:00000001:0.0:1788572317.049429:0:2382:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788572317.049432:0:2382:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572317.049435:0:2382:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572317.049438:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572317.049447:0:2382:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788572317.049449:0:2382:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=5 : 5 : 5) 00000100:00000001:0.0:1788572317.049451:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788572317.049452:0:2382:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788572317.049453:0:2382:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572317.049455:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572317.055081:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788572317.055092:0:2385:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788572317.055095:0:2385:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788572317.055115:0:2385:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada455a4380 x1875449616090752/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572317.055131:0:2385:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788572317.055133:0:2385:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572317.055141:0:2385:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788572317.055146:0:2385:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788572317.055149:0:2385:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788572317.055153:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616090752:192.168.202.113@tcp:400:kworker.0 00000100:00000001:2.0:1788572317.055159:0:2385:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788572317.055166:0:2385:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788572317.055169:0:2385:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572317.055171:0:2385:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788572317.055180:0:2385:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada45575200. 02000000:00000001:2.0:1788572317.055182:0:2385:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788572317.055192:0:2385:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:2.0:1788572317.055198:0:2385:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada43d96cb8. 00000400:00000010:2.0:1788572317.055207:0:2385:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada43c8faa0. 00000100:00000200:2.0:1788572317.055217:0:2385:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616090752, portal 10 00000100:00000001:2.0:1788572317.055220:0:2385:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788572317.055221:0:2385:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860989932416 : -111212719619200 : ffff9ada455a4380) 00000100:00000040:2.0:1788572317.055231:0:2385:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada455a4380 x1875449616090752/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572333 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572317.055240:0:2385:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788572317.055243:0:2385:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:2.0:1788572317.055246:0:2385:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada43c8f110. 00000100:00000200:2.0:1788572317.055250:0:2385:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875449616090752, offset 0 00000400:00000200:2.0:1788572317.055565:0:2385:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:2.0:1788572317.055576:0:2385:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572317.055586:0:2385:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:2.0:1788572317.055592:0:2385:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788572317.055597:0:2385:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:2.0:1788572317.055601:0:2385:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788572317.055604:0:2385:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:2.0:1788572317.055677:0:2385:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22279:22279:255:4294967295] 192.168.202.113@tcp LPNI seq info [22275:22275:7:4294967295] 00000400:00000200:2.0:1788572317.055683:0:2385:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572317.055695:0:2385:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:2.0:1788572317.055703:0:2385:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:2.0:1788572317.055710:0:2385:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada617e8f00. 00000800:00000200:2.0:1788572317.055718:0:2385:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:2.0:1788572317.055727:0:2385:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:2.0:1788572317.055732:0:2385:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada617e8f00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788572317.055737:0:2385:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572317.055743:0:2385:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: using a timeout of 41271 usecs (42 jiffies) 00000400:00000200:2.0:1788572317.055745:0:2385:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:2.0:1788572317.055749:0:2385:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298902066 00000100:00000001:2.0:1788572317.055753:0:2385:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298907469 00000100:00000001:2.0:1788572317.055755:0:2385:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298902066 00000100:00000001:2.0:1788572317.055758:0:2385:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298907469 00000100:00000001:2.0:1788572317.055760:0:2385:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788572317.055764:0:2385:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572317.055768:0:2385:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572317.055772:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572317.055781:0:2385:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788572317.055784:0:2385:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788572317.055786:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788572317.055787:0:2385:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788572317.055789:0:2385:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788572317.055792:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00000001:2.0:1788572317.055793:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572317.055796:0:2385:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788572317.055797:0:2385:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572317.055798:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788572317.055799:0:2385:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788572317.055801:0:2385:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788572317.055806:0:2385:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada455a6680 x1875449616090624/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572317.055814:0:2385:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788572317.055815:0:2385:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572317.055818:0:2385:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788572317.055820:0:2385:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788572317.055821:0:2385:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788572317.055824:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449616090624:192.168.202.113@tcp:400:kworker.0 00000100:00000001:2.0:1788572317.055828:0:2385:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788572317.055831:0:2385:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788572317.055832:0:2385:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572317.055834:0:2385:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788572317.055838:0:2385:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada45574800. 02000000:00000001:2.0:1788572317.055840:0:2385:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788572317.055844:0:2385:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:2.0:1788572317.055846:0:2385:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada43d96a50. 00000400:00000010:2.0:1788572317.055851:0:2385:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada43c8f990. 00000100:00000200:2.0:1788572317.055879:0:2385:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616090624, portal 25 00000100:00000001:2.0:1788572317.055882:0:2385:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788572317.055883:0:2385:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860989941376 : -111212719610240 : ffff9ada455a6680) 00000100:00000040:2.0:1788572317.055889:0:2385:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada455a6680 x1875449616090624/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572333 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572317.055895:0:2385:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788572317.055897:0:2385:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:2.0:1788572317.055900:0:2385:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada43c8f5d8. 00000100:00000200:2.0:1788572317.055902:0:2385:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875449616090624, offset 0 00000400:00000200:2.0:1788572317.055907:0:2385:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:2.0:1788572317.055910:0:2385:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572317.055914:0:2385:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:2.0:1788572317.055917:0:2385:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788572317.055919:0:2385:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:2.0:1788572317.055921:0:2385:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788572317.055923:0:2385:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:2.0:1788572317.055926:0:2385:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22280:22280:254:4294967295] 192.168.202.113@tcp LPNI seq info [22276:22276:6:4294967295] 00000400:00000200:2.0:1788572317.055930:0:2385:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000800:00000200:3.0:1788572317.056812:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788572317.056819:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada5074ad00. 00000400:00000200:3.0:1788572317.056827:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572317.057052:0:2385:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000400:00000200:3.0:1788572317.059662:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:3.0:1788572317.059678:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada5176a330 00000400:00000010:3.0:1788572317.059684:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada5176a330. 00000100:00000001:3.0:1788572317.059693:0:2378:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:3.0:1788572317.059705:0:2378:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada455a7100 x1875449616091008/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572337 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572317.059720:0:2378:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572317.059724:0:2378:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada455a7100 x1875449616091008/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572337 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572317.059731:0:2378:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572317.059733:0:2378:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000200:3.0:1788572317.059831:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788572317.059836:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada617e8f00. 00000400:00000200:3.0:1788572317.059842:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572317.059847:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:3.0:1788572317.059851:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada43c8f110 00000400:00000010:3.0:1788572317.059885:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada43c8f110. 00000100:00000001:3.0:1788572317.059889:0:2378:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:3.0:1788572317.059895:0:2378:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada455a4380 x1875449616090752/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572333 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572317.059905:0:2378:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572317.059908:0:2378:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada455a4380 x1875449616090752/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572333 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572317.059915:0:2378:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572317.059916:0:2378:0:(events.c:69:request_out_callback()) Process leaving 00000100:00000001:3.0:1788572317.059969:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788572317.059974:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788572317.062679:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572317.062703:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572317.062710:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572317.062713:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572317.062725:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572317.062734:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572317.062745:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572317.062751:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572317.062757:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572317.062762:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 4 MB=0x6a9b624294780 00000400:00000200:3.0:1788572317.062767:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572317.062775:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.202.113@tcp of length 224/224 into md 0x356e5 [1] + 0 00000800:00000001:3.0:1788572317.062783:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572317.062790:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572317.062794:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572317.062799:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572317.062809:0:2377:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572317.062819:0:2377:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada455a7100 x1875449616091008/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572337 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572317.062832:0:2377:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=0 replen=544 req@ffff9ada455a7100 x1875449616091008/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572337 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572317.063239:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788572317.063244:0:2382:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788572317.063819:0:2377:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572317.063835:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572317.063842:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572317.065087:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572317.065101:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572317.065105:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572317.065107:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572317.065117:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572317.065126:0:2378:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572317.065135:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572317.065140:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572317.065145:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572317.065150:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 10 MB=0x6a9b624294680 00000400:00000200:3.0:1788572317.065157:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572317.065164:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.113@tcp of length 224/224 into md 0x356ed [1] + 224 00000800:00000001:3.0:1788572317.065171:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572317.065179:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572317.065183:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572317.065189:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572317.065197:0:2378:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572317.065207:0:2378:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada455a4380 x1875449616090752/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572333 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572317.065220:0:2378:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada455a4380 x1875449616090752/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572333 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572317.065230:0:2378:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572317.065237:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572317.065241:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:2.0:1788572317.074535:0:2385:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:2.0:1788572317.074552:0:2385:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada617e8500. 00000800:00000200:2.0:1788572317.074562:0:2385:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:2.0:1788572317.074572:0:2385:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:2.0:1788572317.074577:0:2385:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada617e8500 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788572317.074597:0:2385:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788572317.074618:0:2385:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:2.0:1788572317.074624:0:2385:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS new deadline estimation for cpu 2 is 4298912615 00000100:00000001:2.0:1788572317.074634:0:2385:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4298907469 00000100:00000001:2.0:1788572317.074640:0:2385:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS new deadline estimation for cpu 3 is 4298912615 00000100:00000001:2.0:1788572317.074644:0:2385:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4298907469 00000100:00000001:2.0:1788572317.074647:0:2385:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788572317.074651:0:2385:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572317.074656:0:2385:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788572317.074668:0:2385:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada455a5500 x1875449616090880/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572317.074678:0:2385:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788572317.074680:0:2385:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572317.074685:0:2385:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788572317.074689:0:2385:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788572317.074691:0:2385:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788572317.074695:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616090880:192.168.202.113@tcp:400:kworker.0 00000100:00000001:2.0:1788572317.074700:0:2385:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788572317.074707:0:2385:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788572317.074710:0:2385:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572317.074714:0:2385:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788572317.074722:0:2385:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada45574600. 02000000:00000001:2.0:1788572317.074725:0:2385:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788572317.074731:0:2385:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:2.0:1788572317.074737:0:2385:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada43d967e8. 00000400:00000010:2.0:1788572317.074747:0:2385:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada43c8fcc0. 00000100:00000200:2.0:1788572317.074755:0:2385:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616090880, portal 10 00000100:00000001:2.0:1788572317.074757:0:2385:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788572317.074759:0:2385:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860989936896 : -111212719614720 : ffff9ada455a5500) 00000100:00000040:2.0:1788572317.074768:0:2385:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada455a5500 x1875449616090880/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572333 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572317.074776:0:2385:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788572317.074778:0:2385:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:2.0:1788572317.074781:0:2385:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada43c8ff68. 00000100:00000200:2.0:1788572317.074784:0:2385:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875449616090880, offset 0 00000400:00000200:2.0:1788572317.074790:0:2385:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:2.0:1788572317.074801:0:2385:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572317.074810:0:2385:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:2.0:1788572317.074817:0:2385:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788572317.074821:0:2385:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:2.0:1788572317.074825:0:2385:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788572317.074828:0:2385:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:2.0:1788572317.074833:0:2385:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22281:22281:255:4294967295] 192.168.202.113@tcp LPNI seq info [22277:22277:7:4294967295] 00000400:00000200:2.0:1788572317.074838:0:2385:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572317.074848:0:2385:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:2.0:1788572317.074868:0:2385:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:2.0:1788572317.074872:0:2385:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada617e8f00. 00000800:00000200:2.0:1788572317.074876:0:2385:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:2.0:1788572317.074881:0:2385:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:2.0:1788572317.074884:0:2385:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada617e8f00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788572317.074887:0:2385:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572317.074891:0:2385:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: using a timeout of 16551 usecs (17 jiffies) 00000400:00000200:2.0:1788572317.074894:0:2385:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:2.0:1788572317.074896:0:2385:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298902061 00000100:00000001:2.0:1788572317.074899:0:2385:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298912615 00000100:00000001:2.0:1788572317.074901:0:2385:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298902061 00000100:00000001:2.0:1788572317.074904:0:2385:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298912615 00000100:00000001:2.0:1788572317.074906:0:2385:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788572317.074908:0:2385:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:2.0:1788572317.074915:0:2385:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788572317.074916:0:2385:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788572317.074920:0:2385:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada43d96cb8. 00000400:00000200:2.0:1788572317.074926:0:2385:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada43c8faa0 00000400:00000010:2.0:1788572317.074929:0:2385:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada43c8faa0. 00000100:00000001:2.0:1788572317.074932:0:2385:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:2.0:1788572317.074950:0:2385:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada455a4380 x1875449616090752/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572333 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788572317.074960:0:2385:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada455a4380 x1875449616090752/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572333 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572317.074969:0:2385:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788572317.074970:0:2385:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572317.074973:0:2385:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:2.0:1788572317.074976:0:2385:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788572317.074978:0:2385:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788572317.074981:0:2385:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572317.074983:0:2385:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788572317.074993:0:2385:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada51140460 time=12 v=5 (5 5 5 5) 00000100:00001000:2.0:1788572317.074997:0:2385:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada51140418 time=108 v=5 (2 0 0 0) 00000100:00000001:2.0:1788572317.075000:0:2385:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788572317.075002:0:2385:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572317.075005:0:2385:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788572317.075011:0:2385:0:(ldlm_request.c:1703:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572317.075015:0:2385:0:(client.c:3110:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788572317.075017:0:2385:0:(client.c:3122:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: skip recheck: last_committed 219043334508 00000100:00000001:2.0:1788572317.075019:0:2385:0:(client.c:3123:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788572317.075021:0:2385:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572317.075025:0:2385:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada455a4380 x1875449616090752/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572333 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572317.075047:0:2385:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788572317.075050:0:2385:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788572317.075053:0:2385:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572317.075057:0:2385:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada455a4380 x1875449616090752/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572333 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788572317.075066:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616090752:192.168.202.113@tcp:400:kworker.0 00000100:00000001:2.0:1788572317.075070:0:2385:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572317.075074:0:2385:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572317.075077:0:2385:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455a4380 x1875449616090752/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572333 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572317.075084:0:2385:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572317.075087:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572317.075089:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada45575200. 02000000:00000001:2.0:1788572317.075093:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572317.075096:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572317.075098:0:2385:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=3 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572317.075101:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572317.075104:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada50c31600. 02000000:00000001:2.0:1788572317.075109:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572317.075111:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572317.075113:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455a4380. 00000100:00000001:2.0:1788572317.075117:0:2385:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572317.075118:0:2385:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572317.075122:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572317.075125:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788572317.075128:0:2385:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788572317.075130:0:2385:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572317.075132:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788572317.075769:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788572317.075776:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada617e8500. 00000400:00000200:3.0:1788572317.075782:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572317.075789:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:3.0:1788572317.075796:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada43c8f5d8 00000400:00000010:3.0:1788572317.075798:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada43c8f5d8. 00000100:00000001:3.0:1788572317.075802:0:2377:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:3.0:1788572317.075810:0:2377:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada455a6680 x1875449616090624/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572333 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572317.075820:0:2377:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572317.075823:0:2377:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada455a6680 x1875449616090624/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572333 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572317.075830:0:2377:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572317.075832:0:2377:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000200:3.0:1788572317.075944:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788572317.075947:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada617e8f00. 00000400:00000200:3.0:1788572317.075951:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572317.075954:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:3.0:1788572317.075957:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada43c8ff68 00000400:00000010:3.0:1788572317.075959:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada43c8ff68. 00000100:00000001:3.0:1788572317.075961:0:2377:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:3.0:1788572317.075965:0:2377:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada455a5500 x1875449616090880/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572333 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572317.075973:0:2377:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572317.075976:0:2377:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada455a5500 x1875449616090880/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572333 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572317.075982:0:2377:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572317.075984:0:2377:0:(events.c:69:request_out_callback()) Process leaving 00000400:00000001:0.0:1788572317.076983:0:2382:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788572317.076988:0:2382:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788572317.076992:0:2382:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada60acaf20. 00000400:00000200:0.0:1788572317.076997:0:2382:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada5176a088 00000400:00000010:0.0:1788572317.076999:0:2382:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada5176a088. 00000100:00000001:0.0:1788572317.077003:0:2382:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:0.0:1788572317.077014:0:2382:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada455a7100 x1875449616091008/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572337 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788572317.077024:0:2382:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada455a7100 x1875449616091008/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572337 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572317.077044:0:2382:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788572317.077046:0:2382:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572317.077049:0:2382:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:0.0:1788572317.077052:0:2382:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788572317.077054:0:2382:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788572317.077056:0:2382:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572317.077059:0:2382:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788572317.077069:0:2382:0:(import.c:2195:obd_at_measure()) add 0 to ffff9ada48407c60 time=162 v=5 (5 5 5 5) 00000100:00001000:0.0:1788572317.077074:0:2382:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada48407c18 time=39 v=15 (15 1 1 1) 00000100:00000001:0.0:1788572317.077076:0:2382:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788572317.077078:0:2382:0:(client.c:1506:ptlrpc_check_status()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000001:0.0:1788572317.077081:0:2382:0:(client.c:1712:after_reply()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000040:0.0:1788572317.077086:0:2382:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada455a7100 x1875449616091008/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/224 e 0 to 0 dl 1788572337 ref 1 fl Rpc:RNQU/200/0 rc -19/-19 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572317.077093:0:2382:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=18446744073709551597 : -19 : 0xffffffffffffffed) 00000100:00000001:0.0:1788572317.077096:0:2382:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788572317.077098:0:2382:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572317.077101:0:2382:0:(import.c:1101:ptlrpc_connect_interpret()) Process entered 00000100:00000001:0.0:1788572317.077103:0:2382:0:(import.c:1136:ptlrpc_connect_interpret()) Process leaving via out (rc=18446744073709551597 : -19 : 0xffffffffffffffed) 00000100:00080000:0.0:1788572317.077107:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada48407800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788572317.077112:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.113@tcp failed (-19) 00000100:00000001:0.0:1788572317.077115:0:2382:0:(import.c:1561:ptlrpc_connect_interpret()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000040:0.0:1788572317.077118:0:2382:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada455a7100 x1875449616091008/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/224 e 0 to 0 dl 1788572337 ref 1 fl Interpret:RNQU/200/0 rc -19/-19 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572317.077125:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455a7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616091008:192.168.202.113@tcp:8:kworker.0 00000100:00000001:0.0:1788572317.077130:0:2382:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572317.077133:0:2382:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788572317.077136:0:2382:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455a7100 x1875449616091008/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/224 e 0 to 0 dl 1788572337 ref 1 fl Complete:RNQU/200/0 rc -19/-19 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572317.077142:0:2382:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788572317.077144:0:2382:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788572317.077147:0:2382:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada45653000. 02000000:00000001:0.0:1788572317.077151:0:2382:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788572317.077154:0:2382:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:0.0:1788572317.077155:0:2382:0:(genops.c:1223:class_import_put()) import ffff9ada48407800 refcount=2 obd=lustre-OST0000-osc-ffff9ada58b0a000 00000020:00000001:0.0:1788572317.077158:0:2382:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:0.0:1788572317.077160:0:2382:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 1024 at ffff9ada607bd000. 02000000:00000001:0.0:1788572317.077168:0:2382:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788572317.077170:0:2382:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788572317.077171:0:2382:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455a7100. 00000100:00000001:0.0:1788572317.077175:0:2382:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788572317.077176:0:2382:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572317.077179:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572317.077187:0:2382:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788572317.077190:0:2382:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572317.077192:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788572317.077194:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788572317.078400:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572317.078427:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572317.078433:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572317.078436:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572317.078447:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572317.078455:0:2378:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572317.078466:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572317.078480:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572317.078488:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572317.078493:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 25 MB=0x6a9b624294600 00000400:00000200:3.0:1788572317.078499:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572317.078506:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.202.113@tcp of length 224/224 into md 0x356fd [1] + 224 00000800:00000001:3.0:1788572317.078516:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572317.078523:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572317.078528:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572317.078533:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572317.078543:0:2378:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572317.078552:0:2378:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada455a6680 x1875449616090624/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572333 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572317.078562:0:2378:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada455a6680 x1875449616090624/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572333 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572317.078585:0:2378:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572317.078596:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572317.078599:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572317.079098:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788572317.079104:0:2385:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788572317.079112:0:2385:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788572317.079114:0:2385:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788572317.079119:0:2385:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada43d96a50. 00000400:00000200:2.0:1788572317.079125:0:2385:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada43c8f990 00000400:00000010:2.0:1788572317.079128:0:2385:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada43c8f990. 00000100:00000001:2.0:1788572317.079132:0:2385:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:2.0:1788572317.079139:0:2385:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada455a6680 x1875449616090624/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572333 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788572317.079149:0:2385:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada455a6680 x1875449616090624/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572333 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572317.079155:0:2385:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788572317.079157:0:2385:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572317.079160:0:2385:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:2.0:1788572317.079163:0:2385:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788572317.079166:0:2385:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788572317.079168:0:2385:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572317.079171:0:2385:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788572317.079178:0:2385:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada58a62c60 time=79 v=5 (5 5 5 5) 00000100:00001000:2.0:1788572317.079182:0:2385:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada58a62c18 time=109 v=5 (1 0 0 0) 00000100:00000001:2.0:1788572317.079185:0:2385:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788572317.079187:0:2385:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572317.079190:0:2385:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788572317.079192:0:2385:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572317.079195:0:2385:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572317.079199:0:2385:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada455a6680 x1875449616090624/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572333 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572317.079205:0:2385:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788572317.079208:0:2385:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788572317.079210:0:2385:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572317.079214:0:2385:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada455a6680 x1875449616090624/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572333 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788572317.079221:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449616090624:192.168.202.113@tcp:400:kworker.0 00000100:00000001:2.0:1788572317.079227:0:2385:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572317.079229:0:2385:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572317.079233:0:2385:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455a6680 x1875449616090624/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572333 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572317.079239:0:2385:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572317.079242:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572317.079246:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada45574800. 02000000:00000001:2.0:1788572317.079261:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572317.079265:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572317.079267:0:2385:0:(genops.c:1223:class_import_put()) import ffff9ada58a62800 refcount=3 obd=MGC192.168.202.113@tcp 00000020:00000001:2.0:1788572317.079270:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572317.079272:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada50c31a00. 02000000:00000001:2.0:1788572317.079276:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572317.079278:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572317.079280:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455a6680. 00000100:00000001:2.0:1788572317.079284:0:2385:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572317.079286:0:2385:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572317.079290:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572317.079292:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788572317.079293:0:2385:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788572317.079295:0:2385:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572317.079297:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788572317.080249:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572317.080263:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572317.080268:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572317.080271:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572317.080280:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572317.080286:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572317.080295:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572317.080299:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572317.080305:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572317.080309:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 10 MB=0x6a9b624294700 00000400:00000200:3.0:1788572317.080323:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572317.080329:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.113@tcp of length 224/224 into md 0x3570d [1] + 224 00000800:00000001:3.0:1788572317.080337:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572317.080803:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572317.080809:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572317.080815:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572317.080824:0:2377:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572317.080834:0:2377:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada455a5500 x1875449616090880/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572333 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572317.080846:0:2377:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada455a5500 x1875449616090880/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572333 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572317.080954:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788572317.080958:0:2385:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788572317.080976:0:2377:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572317.080988:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572317.080993:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:2.0:1788572317.086804:0:2385:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788572317.086814:0:2385:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788572317.086820:0:2385:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada43d967e8. 00000400:00000200:2.0:1788572317.086833:0:2385:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada43c8fcc0 00000400:00000010:2.0:1788572317.086836:0:2385:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada43c8fcc0. 00000100:00000001:2.0:1788572317.086842:0:2385:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:2.0:1788572317.086871:0:2385:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada455a5500 x1875449616090880/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572333 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788572317.086884:0:2385:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada455a5500 x1875449616090880/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572333 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572317.086893:0:2385:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788572317.086895:0:2385:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572317.086898:0:2385:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:2.0:1788572317.086902:0:2385:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788572317.086906:0:2385:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788572317.086908:0:2385:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572317.086911:0:2385:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788572317.086923:0:2385:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada58b08460 time=79 v=5 (5 5 5 5) 00000100:00001000:2.0:1788572317.086928:0:2385:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada58b08418 time=79 v=5 (1 1 1 1) 00000100:00000001:2.0:1788572317.086931:0:2385:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788572317.086933:0:2385:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572317.087282:0:2385:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788572317.087291:0:2385:0:(ldlm_request.c:1703:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572317.087296:0:2385:0:(client.c:3110:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788572317.087299:0:2385:0:(client.c:3122:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff9ada58b0a000: skip recheck: last_committed 0 00000100:00000001:2.0:1788572317.087300:0:2385:0:(client.c:3123:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788572317.087302:0:2385:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572317.087309:0:2385:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada455a5500 x1875449616090880/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572333 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572317.087325:0:2385:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788572317.087329:0:2385:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788572317.087332:0:2385:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572317.087337:0:2385:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada455a5500 x1875449616090880/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572333 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788572317.087346:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455a5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616090880:192.168.202.113@tcp:400:kworker.0 00000100:00000001:2.0:1788572317.087352:0:2385:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572317.087550:0:2385:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572317.087556:0:2385:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455a5500 x1875449616090880/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572333 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572317.087755:0:2385:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572317.087758:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572317.087763:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada45574600. 02000000:00000001:2.0:1788572317.087769:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572317.087773:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572317.087774:0:2385:0:(genops.c:1223:class_import_put()) import ffff9ada58b08000 refcount=3 obd=lustre-MDT0001-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572317.087777:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572317.087782:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada50c31400. 02000000:00000001:2.0:1788572317.087786:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572317.087789:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572317.087791:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455a5500. 00000100:00000001:2.0:1788572317.087797:0:2385:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572317.087799:0:2385:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572317.087802:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572317.087812:0:2385:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788572317.087814:0:2385:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572317.087817:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788572317.087821:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572317.559143:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572317.559151:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572317.559159:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572317.559160:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572317.559164:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572317.559166:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572317.559170:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572317.559171:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572317.564842:0:15205:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572317.564852:0:15205:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572317.564889:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572318.583077:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572318.583087:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572318.583095:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572318.583096:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572318.583100:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572318.583101:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572318.583105:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572318.583106:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788572318.583378:0:15204:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788572318.583383:0:15204:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788572318.583386:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572319.607021:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572319.607030:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572319.607038:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572319.607039:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572319.607043:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572319.607044:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572319.607049:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572319.607050:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572319.607156:0:33423:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572319.607161:0:33423:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572319.607166:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572320.631172:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572320.631182:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572320.631191:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572320.631193:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572320.631197:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572320.631198:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572320.631203:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572320.631204:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572320.631339:0:15205:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572320.631342:0:15205:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572320.631346:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572321.655560:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572321.655568:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572321.655575:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572321.655576:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572321.655580:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572321.655584:0:365:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:2.0:1788572321.655586:0:365:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:2.0:1788572321.655594:0:365:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:2.0:1788572321.655596:0:365:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:2.0:1788572321.655597:0:365:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572321.655601:0:365:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:2.0:1788572321.655604:0:365:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572321.655606:0:365:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572321.655608:0:365:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572321.655611:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572321.655612:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788572321.655700:0:15204:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788572321.655704:0:15204:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788572321.655707:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788572322.167893:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788572322.167913:0:77466:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788572322.167923:0:77466:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada456bdf80. 00000020:00000040:1.0:1788572322.167931:0:77466:0:(genops.c:1210:class_import_get()) import ffff9ada58a62800 refcount=4 obd=MGC192.168.202.113@tcp 00000100:00000001:1.0:1788572322.167936:0:77466:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788572322.167939:0:77466:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788572322.167945:0:77466:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788572322.167955:0:77466:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada48e6f100. 00000100:00000001:1.0:1788572322.167964:0:77466:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572322.167981:0:77466:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS req@ffff9ada456bdf80 x1875449616091136/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788572322.167993:0:77466:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788572322.167998:0:77466:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572322.168006:0:77466:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000fffb7165] to pc [ptlrpcd_00_00+0] req@ffff9ada456bdf80 x1875449616091136/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572322.168032:0:77466:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572322.168034:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788572322.168038:0:77466:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788572322.168041:0:77466:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada456bce00. 00000020:00000040:1.0:1788572322.168044:0:77466:0:(genops.c:1210:class_import_get()) import ffff9ada51140000 refcount=4 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000100:00000001:1.0:1788572322.168046:0:77466:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788572322.168047:0:77466:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788572322.168049:0:77466:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788572322.168053:0:77466:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada48e6fa00. 00000100:00000001:1.0:1788572322.168057:0:77466:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572322.168061:0:77466:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID req@ffff9ada456bce00 x1875449616091264/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788572322.168068:0:77466:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788572322.168070:0:77466:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572322.168073:0:77466:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000000e2f9a7e] to pc [ptlrpcd_00_01+1] req@ffff9ada456bce00 x1875449616091264/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572322.168081:0:77466:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572322.168083:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788572322.168087:0:77466:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788572322.168089:0:77466:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada456bea00. 00000020:00000040:1.0:1788572322.168092:0:77466:0:(genops.c:1210:class_import_get()) import ffff9ada58b08000 refcount=4 obd=lustre-MDT0001-mdc-ffff9ada58b0a000 00000100:00000001:1.0:1788572322.168094:0:77466:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788572322.168095:0:77466:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788572322.168097:0:77466:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788572322.168100:0:77466:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada48e6f400. 00000100:00000001:1.0:1788572322.168102:0:77466:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572322.168106:0:77466:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID req@ffff9ada456bea00 x1875449616091392/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788572322.168112:0:77466:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788572322.168114:0:77466:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572322.168117:0:77466:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000017fbc1c1] to pc [ptlrpcd_00_00+0] req@ffff9ada456bea00 x1875449616091392/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572322.168136:0:77466:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00080000:1.0:1788572322.168139:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572322.168142:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00000001:1.0:1788572322.168145:0:77466:0:(import.c:745:ptlrpc_connect_import_locked()) Process entered 00000100:00080000:1.0:1788572322.168147:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada48407800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00000001:1.0:1788572322.168151:0:77466:0:(import.c:520:import_select_connection()) Process entered 00000100:00080000:1.0:1788572322.168156:0:77466:0:(import.c:545:import_select_connection()) lustre-OST0000-osc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4234 00000400:00000200:1.0:1788572322.168164:0:77466:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572322.168170:0:77466:0:(peer.c:1689:LNetPeerDiscovered()) Peer NID 192.168.202.113@tcp is discovered: rc = 1 00000100:00001000:1.0:1788572322.168174:0:77466:0:(import.c:2195:obd_at_measure()) add 20 to ffff9ada48407c18 time=44 v=15 (15 1 1 1) 00000100:00001000:1.0:1788572322.168177:0:77466:0:(import.c:2261:obd_at_measure()) AT ffff9ada48407c18 change: old=15 new=20 delta=5 (val=20) hist 20 1 1 1 00000100:00080000:1.0:1788572322.168179:0:77466:0:(import.c:630:import_select_connection()) lustre-OST0000-osc-ffff9ada58b0a000: tried all connections, increasing latency to 15s 00000100:00000040:1.0:1788572322.168182:0:77466:0:(lustre_net.h:1977:ptlrpc_connection_put()) PUT conn=ffff9ada51db1200 refcount 14 to 192.168.202.113@tcp 00000100:00000001:1.0:1788572322.168185:0:77466:0:(connection.c:157:ptlrpc_connection_addref()) Process entered 00000100:00000040:1.0:1788572322.168187:0:77466:0:(connection.c:162:ptlrpc_connection_addref()) conn=ffff9ada51db1200 refcount 15 to 192.168.202.113@tcp 00000100:00000001:1.0:1788572322.168189:0:77466:0:(connection.c:164:ptlrpc_connection_addref()) Process leaving (rc=18446632861199700480 : -111212509851136 : ffff9ada51db1200) 00000020:00000001:1.0:1788572322.168192:0:77466:0:(genops.c:792:class_conn2export()) Process entered 00000020:00000040:1.0:1788572322.168194:0:77466:0:(genops.c:804:class_conn2export()) looking for export cookie 0xef30c340bfb39c83 00000020:00000001:1.0:1788572322.168197:0:77466:0:(lustre_handles.c:130:class_handle2object()) Process entered 00000020:00000040:1.0:1788572322.168200:0:77466:0:(lustre_handles.c:148:class_handle2object()) GET export ffff9ada50c3b800 refcount=4 00000020:00000001:1.0:1788572322.168202:0:77466:0:(lustre_handles.c:155:class_handle2object()) Process leaving (rc=18446632861181392896 : -111212528158720 : ffff9ada50c3b800) 00000020:00000001:1.0:1788572322.168205:0:77466:0:(genops.c:806:class_conn2export()) Process leaving (rc=18446632861181392896 : -111212528158720 : ffff9ada50c3b800) 00000100:00000040:1.0:1788572322.168208:0:77466:0:(lustre_net.h:1977:ptlrpc_connection_put()) PUT conn=ffff9ada51db1200 refcount 14 to 192.168.202.113@tcp 00000100:00000001:1.0:1788572322.168210:0:77466:0:(connection.c:157:ptlrpc_connection_addref()) Process entered 00000100:00000040:1.0:1788572322.168212:0:77466:0:(connection.c:162:ptlrpc_connection_addref()) conn=ffff9ada51db1200 refcount 15 to 192.168.202.113@tcp 00000100:00000001:1.0:1788572322.168214:0:77466:0:(connection.c:164:ptlrpc_connection_addref()) Process leaving (rc=18446632861199700480 : -111212509851136 : ffff9ada51db1200) 00000020:00000040:1.0:1788572322.168217:0:77466:0:(genops.c:879:class_export_put()) PUTting export ffff9ada50c3b800 : new refcount 3 00000100:00080000:1.0:1788572322.168221:0:77466:0:(import.c:665:import_select_connection()) lustre-OST0000-osc-ffff9ada58b0a000: import ffff9ada48407800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00000001:1.0:1788572322.168224:0:77466:0:(import.c:672:import_select_connection()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572322.168226:0:77466:0:(sec.c:1579:sptlrpc_import_sec_adapt()) Process entered 02000000:00000001:1.0:1788572322.168234:0:77466:0:(sec.c:1619:sptlrpc_import_sec_adapt()) Process leaving via out (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788572322.168237:0:77466:0:(sec.c:1647:sptlrpc_import_sec_adapt()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572322.168239:0:77466:0:(obd_class.h:888:obd_reconnect()) Process entered 00000008:00100000:1.0:1788572322.168245:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683394 ocd_grant: 8388608, lost: 0. 00000008:00000001:1.0:1788572322.168247:0:77466:0:(osc_request.c:3833:osc_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572322.168249:0:77466:0:(obd_class.h:902:obd_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788572322.168255:0:77466:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada456bca80. 00000020:00000040:1.0:1788572322.168258:0:77466:0:(genops.c:1210:class_import_get()) import ffff9ada48407800 refcount=3 obd=lustre-OST0000-osc-ffff9ada58b0a000 02000000:00000001:1.0:1788572322.168261:0:77466:0:(sec.c:2038:sptlrpc_sepol_get()) Process entered 02000000:00000001:1.0:1788572322.168263:0:77466:0:(sec.c:2048:sptlrpc_sepol_get()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572322.168265:0:77466:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788572322.168267:0:77466:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788572322.168269:0:77466:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788572322.168274:0:77466:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 1024 at ffff9ada454bbc00. 00000100:00000001:1.0:1788572322.168277:0:77466:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788572322.168283:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada456bca80 x1875449616091520/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788572322.168290:0:77466:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788572322.168292:0:77466:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572322.168296:0:77466:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000005f2d402f] to pc [ptlrpcd_rcv+-1] req@ffff9ada456bca80 x1875449616091520/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572322.168316:0:77466:0:(import.c:877:ptlrpc_connect_import_locked()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572322.168320:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1788572322.168324:0:77466:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:1.0:1788572322.168330:0:77466:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (4244) 00000100:00000001:0.0:1788572322.168671:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788572322.168675:0:2383:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788572322.168678:0:2383:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788572322.168685:0:2383:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada456bdf80 x1875449616091136/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572322.168693:0:2383:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788572322.168696:0:2383:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572322.168700:0:2383:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788572322.168703:0:2383:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788572322.168705:0:2383:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788572322.168709:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada456bdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449616091136:192.168.202.113@tcp:400:kworker.0 00000100:00000001:0.0:1788572322.168713:0:2383:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788572322.168721:0:2383:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788572322.168723:0:2383:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572322.168726:0:2383:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788572322.168731:0:2383:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada456bb800. 02000000:00000001:0.0:1788572322.168734:0:2383:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788572322.168740:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:0.0:1788572322.168744:0:2383:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada60acaf20. 00000400:00000010:0.0:1788572322.168752:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada5176a088. 00000100:00000200:0.0:1788572322.168762:0:2383:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616091136, portal 25 00000100:00000001:0.0:1788572322.168765:0:2383:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788572322.168767:0:2383:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860991086464 : -111212718465152 : ffff9ada456bdf80) 00000100:00000040:0.0:1788572322.168773:0:2383:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada456bdf80 x1875449616091136/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572338 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572322.168781:0:2383:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788572322.168784:0:2383:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:0.0:1788572322.168786:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada5176a660. 00000100:00000200:0.0:1788572322.168791:0:2383:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875449616091136, offset 0 00000400:00000200:0.0:1788572322.168798:0:2383:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:0.0:1788572322.168809:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:0.0:1788572322.168817:0:2383:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:0.0:1788572322.168825:0:2383:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788572322.168829:0:2383:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:0.0:1788572322.168834:0:2383:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788572322.168837:0:2383:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:0.0:1788572322.168841:0:2383:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22282:22282:256:4294967295] 192.168.202.113@tcp LPNI seq info [22278:22278:8:4294967295] 00000400:00000200:0.0:1788572322.168846:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:0.0:1788572322.168890:0:2383:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:0.0:1788572322.168908:0:2383:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:0.0:1788572322.168914:0:2383:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada5074aa00. 00000800:00000200:0.0:1788572322.168921:0:2383:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:0.0:1788572322.168929:0:2383:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:0.0:1788572322.168935:0:2383:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada5074aa00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788572322.168947:0:2383:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788572322.168951:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:0.0:1788572322.168955:0:2383:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS new deadline estimation for cpu 2 is 4298917709 00000100:00000001:0.0:1788572322.168962:0:2383:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4298912615 00000100:00000001:0.0:1788572322.168967:0:2383:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS new deadline estimation for cpu 3 is 4298917709 00000100:00000001:0.0:1788572322.168970:0:2383:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4298912615 00000100:00000001:0.0:1788572322.168974:0:2383:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788572322.168977:0:2383:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572322.168980:0:2383:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788572322.168984:0:2383:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada456bea00 x1875449616091392/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572322.168991:0:2383:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788572322.168992:0:2383:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572322.168994:0:2383:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788572322.168996:0:2383:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788572322.168999:0:2383:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788572322.169001:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada456bea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616091392:192.168.202.113@tcp:400:kworker.0 00000100:00000001:0.0:1788572322.169005:0:2383:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788572322.169007:0:2383:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788572322.169009:0:2383:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572322.169010:0:2383:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788572322.169013:0:2383:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada456bb400. 02000000:00000001:0.0:1788572322.169015:0:2383:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788572322.169018:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:0.0:1788572322.169021:0:2383:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada60acacb8. 00000400:00000010:0.0:1788572322.169024:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada5176a770. 00000100:00000200:0.0:1788572322.169027:0:2383:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616091392, portal 10 00000100:00000001:0.0:1788572322.169029:0:2383:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788572322.169030:0:2383:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860991089152 : -111212718462464 : ffff9ada456bea00) 00000100:00000040:0.0:1788572322.169037:0:2383:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada456bea00 x1875449616091392/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572338 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572322.169043:0:2383:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788572322.169045:0:2383:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:0.0:1788572322.169048:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada5176a2a8. 00000100:00000200:0.0:1788572322.169050:0:2383:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875449616091392, offset 0 00000400:00000200:0.0:1788572322.169054:0:2383:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:0.0:1788572322.169057:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:0.0:1788572322.169061:0:2383:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:0.0:1788572322.169063:0:2383:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788572322.169066:0:2383:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:0.0:1788572322.169068:0:2383:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788572322.169071:0:2383:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:0.0:1788572322.169074:0:2383:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22283:22283:255:4294967295] 192.168.202.113@tcp LPNI seq info [22279:22279:7:4294967295] 00000400:00000200:0.0:1788572322.169077:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:0.0:1788572322.169083:0:2383:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:0.0:1788572322.169087:0:2383:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:0.0:1788572322.169091:0:2383:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada5074a800. 00000800:00000200:0.0:1788572322.169094:0:2383:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:0.0:1788572322.169098:0:2383:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:0.0:1788572322.169100:0:2383:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada5074a800 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788572322.169103:0:2383:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572322.169106:0:2383:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: using a timeout of 16587 usecs (17 jiffies) 00000400:00000200:0.0:1788572322.169109:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:0.0:1788572322.169111:0:2383:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298907154 00000100:00000001:0.0:1788572322.169113:0:2383:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298917709 00000100:00000001:0.0:1788572322.169115:0:2383:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298907154 00000100:00000001:0.0:1788572322.169117:0:2383:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298917709 00000100:00000001:0.0:1788572322.169130:0:2383:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788572322.169132:0:2383:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572322.169135:0:2383:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572322.169138:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572322.169148:0:2383:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788572322.169150:0:2383:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788572322.169152:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788572322.169154:0:2383:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788572322.169174:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788572322.169178:0:2382:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788572322.169180:0:2382:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788572322.169188:0:2382:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada456bca80 x1875449616091520/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572322.169198:0:2382:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788572322.169200:0:2382:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788572322.169203:0:2382:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788572322.169204:0:2382:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788572322.169207:0:2382:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788572322.169211:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada456bca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616091520:192.168.202.113@tcp:8:kworker.0 00000100:00000001:3.0:1788572322.169215:0:2382:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788572322.169217:0:2382:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788572322.169219:0:2382:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788572322.169221:0:2382:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788572322.169226:0:2382:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at ffff9ada607be800. 02000000:00000001:3.0:1788572322.169229:0:2382:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788572322.169233:0:2382:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000800:00000200:2.0:1788572322.169235:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000400:00000010:3.0:1788572322.169238:0:2382:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada510c92c0. 00000800:00000010:2.0:1788572322.169241:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada5074aa00. 00000400:00000010:3.0:1788572322.169244:0:2382:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada51269b28. 00000100:00000200:3.0:1788572322.169249:0:2382:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1875449616091520, portal 4 00000400:00000200:2.0:1788572322.169251:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1788572322.169252:0:2382:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788572322.169255:0:2382:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860991081088 : -111212718470528 : ffff9ada456bca80) 00000400:00000200:2.0:1788572322.169261:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000040:3.0:1788572322.169265:0:2382:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada456bca80 x1875449616091520/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572347 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788572322.169270:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada5176a660 00000400:00000010:2.0:1788572322.169274:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada5176a660. 00000100:00000001:3.0:1788572322.169278:0:2382:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:2.0:1788572322.169282:0:2378:0:(events.c:35:request_out_callback()) Process entered 00000100:00000040:3.0:1788572322.169283:0:2382:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:3.0:1788572322.169287:0:2382:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada51269f68. 00000100:00000200:2.0:1788572322.169290:0:2378:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada456bdf80 x1875449616091136/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572338 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788572322.169292:0:2382:0:(niobuf.c:94:ptl_send_buf()) Sending 520 bytes to portal 28, xid 1875449616091520, offset 0 00000400:00000200:3.0:1788572322.169303:0:2382:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000100:00000001:2.0:1788572322.169306:0:2378:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000400:00000200:3.0:1788572322.169309:0:2382:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000040:2.0:1788572322.169313:0:2378:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada456bdf80 x1875449616091136/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572338 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788572322.169317:0:2382:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:3.0:1788572322.169324:0:2382:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:2.0:1788572322.169328:0:2378:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788572322.169329:0:2382:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000100:00000001:2.0:1788572322.169333:0:2378:0:(events.c:69:request_out_callback()) Process leaving 00000400:00000200:3.0:1788572322.169571:0:2382:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788572322.169576:0:2382:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:3.0:1788572322.169579:0:2382:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22284:22284:254:4294967295] 192.168.202.113@tcp LPNI seq info [22280:22280:6:4294967295] 00000400:00000200:3.0:1788572322.169584:0:2382:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572322.169590:0:2382:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:3.0:1788572322.169595:0:2382:0:(socklnd_cb.c:931:ksocknal_send()) sending 520 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:3.0:1788572322.169600:0:2382:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada50c31700. 00000800:00000200:3.0:1788572322.169604:0:2382:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:3.0:1788572322.169610:0:2382:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:3.0:1788572322.169613:0:2382:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada50c31700 type 1, nob 616 niov 1 nkiov 1 00000100:00000001:3.0:1788572322.169630:0:2382:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572322.169634:0:2382:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: using a timeout of 18876 usecs (19 jiffies) 00000400:00000200:3.0:1788572322.169637:0:2382:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:3.0:1788572322.169639:0:2382:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298907157 00000100:00000001:3.0:1788572322.169642:0:2382:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298917709 00000100:00000001:3.0:1788572322.169644:0:2382:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298907157 00000100:00000001:3.0:1788572322.169647:0:2382:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298917709 00000100:00000001:3.0:1788572322.169649:0:2382:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788572322.169651:0:2382:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572322.169653:0:2382:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572322.169655:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788572322.169660:0:2382:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788572322.169662:0:2382:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=5 : 5 : 5) 00000100:00000001:3.0:1788572322.169664:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788572322.169665:0:2382:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788572322.169667:0:2382:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572322.169669:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788572322.171085:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788572322.171097:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada5074a800. 00000400:00000200:2.0:1788572322.171105:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572322.171112:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:2.0:1788572322.171129:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada5176a2a8 00000400:00000010:2.0:1788572322.171132:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada5176a2a8. 00000100:00000001:2.0:1788572322.171137:0:2378:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:2.0:1788572322.171146:0:2378:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada456bea00 x1875449616091392/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572338 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572322.171159:0:2378:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572322.171162:0:2378:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada456bea00 x1875449616091392/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572338 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572322.171169:0:2378:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572322.171171:0:2378:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000200:2.0:1788572322.171250:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788572322.171253:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada50c31700. 00000400:00000200:2.0:1788572322.171257:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572322.171260:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:2.0:1788572322.171263:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada51269f68 00000400:00000010:2.0:1788572322.171265:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada51269f68. 00000100:00000001:2.0:1788572322.171268:0:2378:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:2.0:1788572322.171272:0:2378:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada456bca80 x1875449616091520/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572347 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572322.171280:0:2378:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572322.171282:0:2378:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada456bca80 x1875449616091520/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572347 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572322.171289:0:2378:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572322.171290:0:2378:0:(events.c:69:request_out_callback()) Process leaving 00000100:00000001:1.0:1788572322.172572:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572322.172581:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788572322.172584:0:2384:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788572322.172597:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada456bce00 x1875449616091264/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572322.172610:0:2384:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788572322.172612:0:2384:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572322.172617:0:2384:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788572322.172621:0:2384:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788572322.172624:0:2384:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788572322.172628:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada456bce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616091264:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572322.172634:0:2384:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788572322.172641:0:2384:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788572322.172644:0:2384:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572322.172647:0:2384:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788572322.172653:0:2384:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada607e1000. 02000000:00000001:1.0:1788572322.172655:0:2384:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788572322.172664:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:1.0:1788572322.172669:0:2384:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada600d1268. 00000400:00000010:1.0:1788572322.172676:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d23b8. 00000100:00000200:1.0:1788572322.172684:0:2384:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616091264, portal 10 00000100:00000001:1.0:1788572322.172686:0:2384:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788572322.172688:0:2384:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860991081984 : -111212718469632 : ffff9ada456bce00) 00000100:00000040:1.0:1788572322.172698:0:2384:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada456bce00 x1875449616091264/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572338 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572322.172706:0:2384:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788572322.172710:0:2384:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:1.0:1788572322.172713:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2908. 00000100:00000200:1.0:1788572322.172717:0:2384:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875449616091264, offset 0 00000400:00000200:1.0:1788572322.172725:0:2384:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:1.0:1788572322.172735:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572322.172744:0:2384:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:1.0:1788572322.172751:0:2384:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788572322.172756:0:2384:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:1.0:1788572322.172760:0:2384:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788572322.172764:0:2384:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:1.0:1788572322.172768:0:2384:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22285:22285:256:4294967295] 192.168.202.113@tcp LPNI seq info [22281:22281:8:4294967295] 00000400:00000200:1.0:1788572322.172773:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572322.172784:0:2384:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:1.0:1788572322.172791:0:2384:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:1.0:1788572322.172795:0:2384:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada48e6fc00. 00000800:00000200:1.0:1788572322.172803:0:2384:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:1.0:1788572322.172812:0:2384:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:1.0:1788572322.172817:0:2384:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada48e6fc00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788572322.172843:0:2384:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572322.172848:0:2384:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: using a timeout of 41270 usecs (42 jiffies) 00000800:00000200:3.0:1788572322.173052:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788572322.173057:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada48e6fc00. 00000400:00000200:3.0:1788572322.173063:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572322.173068:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:3.0:1788572322.173074:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2908 00000400:00000010:3.0:1788572322.173077:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2908. 00000100:00000001:3.0:1788572322.173082:0:2377:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:3.0:1788572322.173087:0:2377:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada456bce00 x1875449616091264/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572338 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572322.173097:0:2377:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572322.173100:0:2377:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada456bce00 x1875449616091264/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572338 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572322.173106:0:2377:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572322.173108:0:2377:0:(events.c:69:request_out_callback()) Process leaving 00000400:00000200:1.0:1788572322.174727:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:1.0:1788572322.174736:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298907185 00000100:00000001:1.0:1788572322.174742:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298917709 00000100:00000001:1.0:1788572322.174745:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298907185 00000100:00000001:1.0:1788572322.174747:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298917709 00000100:00000001:1.0:1788572322.174750:0:2384:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572322.174754:0:2384:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572322.174758:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572322.174762:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572322.174772:0:2384:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788572322.174775:0:2384:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788572322.174778:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572322.174780:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788572322.174782:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572322.174785:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572322.174873:0:2383:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572322.174879:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788572322.193509:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572322.193539:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788572322.193545:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572322.193548:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572322.193560:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572322.193571:0:2378:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:2.0:1788572322.193587:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572322.193593:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572322.193600:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572322.193605:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 10 MB=0x6a9b624294900 00000400:00000200:2.0:1788572322.193611:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572322.193623:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.113@tcp of length 224/224 into md 0x35725 [1] + 224 00000800:00000001:2.0:1788572322.193633:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572322.193637:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572322.193642:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572322.193646:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:2.0:1788572322.193657:0:2378:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:2.0:1788572322.193668:0:2378:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada456bea00 x1875449616091392/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572338 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788572322.193681:0:2378:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada456bea00 x1875449616091392/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572338 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572322.193695:0:2378:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788572322.193708:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572322.193711:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788572322.193713:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572322.193716:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572322.193719:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572322.193722:0:2378:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:2.0:1788572322.193726:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572322.193729:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572322.193731:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572322.193733:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 10 MB=0x6a9b624294880 00000400:00000200:2.0:1788572322.193736:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572322.193740:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.113@tcp of length 224/224 into md 0x3573d [1] + 224 00000800:00000001:2.0:1788572322.193743:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572322.193746:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572322.193749:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572322.193751:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:2.0:1788572322.193754:0:2378:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:2.0:1788572322.193758:0:2378:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada456bce00 x1875449616091264/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572338 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788572322.193765:0:2378:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada456bce00 x1875449616091264/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572338 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572322.193836:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572322.193840:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788572322.193878:0:2378:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788572322.193884:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572322.193889:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788572322.193891:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572322.193893:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572322.193896:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572322.193900:0:2378:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:2.0:1788572322.193904:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572322.193907:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572322.193910:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572322.193913:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 4 MB=0x6a9b624294980 00000400:00000200:2.0:1788572322.193917:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572322.193923:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.202.113@tcp of length 224/224 into md 0x3572d [1] + 0 00000800:00000001:2.0:1788572322.193927:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572322.193937:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572322.193939:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572322.193941:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:2.0:1788572322.193945:0:2378:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:2.0:1788572322.193953:0:2378:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada456bca80 x1875449616091520/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572347 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788572322.193961:0:2378:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=0 replen=544 req@ffff9ada456bca80 x1875449616091520/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572347 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572322.193977:0:2378:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572322.194022:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:2.0:1788572322.194026:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788572322.194030:0:2382:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000800:00000001:3.0:1788572322.194031:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572322.194035:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572322.194038:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:2.0:1788572322.194041:0:2382:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:3.0:1788572322.194044:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000001:2.0:1788572322.194045:0:2382:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788572322.194051:0:2382:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada510c92c0. 00000400:00000200:3.0:1788572322.194051:0:2378:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:2.0:1788572322.194059:0:2382:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada51269b28 00000400:00000200:3.0:1788572322.194060:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:2.0:1788572322.194064:0:2382:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada51269b28. 00000400:00000200:3.0:1788572322.194065:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:2.0:1788572322.194071:0:2382:0:(events.c:80:reply_in_callback()) Process entered 00000400:00000200:3.0:1788572322.194072:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000200:2.0:1788572322.194078:0:2382:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada456bca80 x1875449616091520/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572347 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788572322.194087:0:2382:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada456bca80 x1875449616091520/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572347 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1788572322.194090:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 25 MB=0x6a9b624294800 00000400:00000200:3.0:1788572322.194098:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:2.0:1788572322.194106:0:2382:0:(events.c:165:reply_in_callback()) Process leaving 00000400:00000200:3.0:1788572322.194106:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.202.113@tcp of length 224/224 into md 0x3571d [1] + 224 00000100:00000001:2.0:1788572322.194110:0:2382:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572322.194113:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:2.0:1788572322.194114:0:2382:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:2.0:1788572322.194120:0:2382:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788572322.194123:0:2382:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788572322.194125:0:2382:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572322.194128:0:2382:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788572322.194140:0:2382:0:(import.c:2195:obd_at_measure()) add 0 to ffff9ada48407c60 time=167 v=5 (5 5 5 5) 00000100:00001000:2.0:1788572322.194145:0:2382:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada48407c18 time=44 v=20 (20 1 1 1) 00000100:00000001:2.0:1788572322.194148:0:2382:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788572322.194151:0:2382:0:(client.c:1506:ptlrpc_check_status()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000001:2.0:1788572322.194154:0:2382:0:(client.c:1712:after_reply()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000040:2.0:1788572322.194159:0:2382:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada456bca80 x1875449616091520/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/224 e 0 to 0 dl 1788572347 ref 1 fl Rpc:RNQU/200/0 rc -19/-19 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572322.194166:0:2382:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=18446744073709551597 : -19 : 0xffffffffffffffed) 00000100:00000001:2.0:1788572322.194170:0:2382:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788572322.194172:0:2382:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572322.194175:0:2382:0:(import.c:1101:ptlrpc_connect_interpret()) Process entered 00000100:00000001:2.0:1788572322.194178:0:2382:0:(import.c:1136:ptlrpc_connect_interpret()) Process leaving via out (rc=18446744073709551597 : -19 : 0xffffffffffffffed) 00000100:00080000:2.0:1788572322.194182:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada48407800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788572322.194187:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.113@tcp failed (-19) 00000100:00000001:2.0:1788572322.194190:0:2382:0:(import.c:1561:ptlrpc_connect_interpret()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000040:2.0:1788572322.194195:0:2382:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada456bca80 x1875449616091520/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/224 e 0 to 0 dl 1788572347 ref 1 fl Interpret:RNQU/200/0 rc -19/-19 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788572322.194201:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada456bca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616091520:192.168.202.113@tcp:8:kworker.0 00000100:00000001:2.0:1788572322.194207:0:2382:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572322.194210:0:2382:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572322.194213:0:2382:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada456bca80 x1875449616091520/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/224 e 0 to 0 dl 1788572347 ref 1 fl Complete:RNQU/200/0 rc -19/-19 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572322.194220:0:2382:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572322.194223:0:2382:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572322.194227:0:2382:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada607be800. 02000000:00000001:2.0:1788572322.194232:0:2382:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572322.194235:0:2382:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572322.194237:0:2382:0:(genops.c:1223:class_import_put()) import ffff9ada48407800 refcount=2 obd=lustre-OST0000-osc-ffff9ada58b0a000 00000020:00000001:2.0:1788572322.194240:0:2382:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572322.194243:0:2382:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 1024 at ffff9ada454bbc00. 02000000:00000001:2.0:1788572322.194246:0:2382:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572322.194249:0:2382:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572322.194251:0:2382:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada456bca80. 00000100:00000001:2.0:1788572322.194254:0:2382:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572322.194255:0:2382:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572322.194258:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572322.194266:0:2382:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788572322.194269:0:2382:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572322.194270:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788572322.194272:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788572322.194775:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572322.194781:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572322.194787:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572322.194793:0:2378:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572322.194799:0:2378:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada456bdf80 x1875449616091136/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572338 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572322.194810:0:2378:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada456bdf80 x1875449616091136/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572338 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572322.194819:0:2378:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572322.194825:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572322.194830:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572322.195186:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788572322.195190:0:2383:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788572322.195196:0:2383:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788572322.195198:0:2383:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788572322.195201:0:2383:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada60acaf20. 00000400:00000200:0.0:1788572322.195205:0:2383:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada5176a088 00000400:00000010:0.0:1788572322.195208:0:2383:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada5176a088. 00000100:00000001:0.0:1788572322.195212:0:2383:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:0.0:1788572322.195218:0:2383:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada456bdf80 x1875449616091136/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572338 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788572322.195226:0:2383:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada456bdf80 x1875449616091136/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572338 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572322.195232:0:2383:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788572322.195233:0:2383:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572322.195236:0:2383:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:0.0:1788572322.195238:0:2383:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788572322.195240:0:2383:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788572322.195242:0:2383:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572322.195243:0:2383:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788572322.195248:0:2383:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada58a62c60 time=84 v=5 (5 5 5 5) 00000100:00001000:0.0:1788572322.195252:0:2383:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada58a62c18 time=114 v=5 (1 0 0 0) 00000100:00000001:0.0:1788572322.195254:0:2383:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788572322.195256:0:2383:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788572322.195259:0:2383:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788572322.195261:0:2383:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572322.195264:0:2383:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572322.195268:0:2383:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada456bdf80 x1875449616091136/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572338 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572322.195275:0:2383:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788572322.195279:0:2383:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788572322.195280:0:2383:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788572322.195284:0:2383:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada456bdf80 x1875449616091136/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572338 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572322.195292:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada456bdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449616091136:192.168.202.113@tcp:400:kworker.0 00000400:00000001:0.0:1788572322.195299:0:2383:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788572322.195300:0:2383:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788572322.195302:0:2383:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada60acacb8. 00000400:00000200:0.0:1788572322.195305:0:2383:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada5176a770 00000400:00000010:0.0:1788572322.195307:0:2383:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada5176a770. 00000100:00000001:0.0:1788572322.195308:0:2383:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:0.0:1788572322.195312:0:2383:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada456bea00 x1875449616091392/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572338 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788572322.195320:0:2383:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada456bea00 x1875449616091392/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572338 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572322.195325:0:2383:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788572322.195327:0:2383:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572322.195328:0:2383:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:0.0:1788572322.195330:0:2383:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788572322.195331:0:2383:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788572322.195332:0:2383:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572322.195334:0:2383:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788572322.195338:0:2383:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada58b08460 time=84 v=5 (5 5 5 5) 00000100:00001000:0.0:1788572322.195340:0:2383:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada58b08418 time=84 v=5 (1 1 1 1) 00000100:00000001:0.0:1788572322.195342:0:2383:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788572322.195344:0:2383:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788572322.195345:0:2383:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788572322.195350:0:2383:0:(ldlm_request.c:1703:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572322.195353:0:2383:0:(client.c:3110:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788572322.195355:0:2383:0:(client.c:3122:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff9ada58b0a000: skip recheck: last_committed 0 00000100:00000001:0.0:1788572322.195357:0:2383:0:(client.c:3123:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788572322.195358:0:2383:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572322.195361:0:2383:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada456bea00 x1875449616091392/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572338 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572322.195379:0:2383:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788572322.195381:0:2383:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788572322.195382:0:2383:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788572322.195385:0:2383:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada456bea00 x1875449616091392/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572338 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572322.195392:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada456bea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616091392:192.168.202.113@tcp:400:kworker.0 00000100:00000001:0.0:1788572322.195395:0:2383:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572322.195398:0:2383:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788572322.195400:0:2383:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada456bdf80 x1875449616091136/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572338 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572322.195406:0:2383:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788572322.195408:0:2383:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788572322.195410:0:2383:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada456bb800. 02000000:00000001:0.0:1788572322.195414:0:2383:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788572322.195417:0:2383:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:0.0:1788572322.195419:0:2383:0:(genops.c:1223:class_import_put()) import ffff9ada58a62800 refcount=3 obd=MGC192.168.202.113@tcp 00000020:00000001:0.0:1788572322.195421:0:2383:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:0.0:1788572322.195423:0:2383:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada48e6f100. 02000000:00000001:0.0:1788572322.195426:0:2383:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788572322.195428:0:2383:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788572322.195429:0:2383:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada456bdf80. 00000100:00000001:0.0:1788572322.195432:0:2383:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788572322.195433:0:2383:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572322.195435:0:2383:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788572322.195437:0:2383:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada456bea00 x1875449616091392/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572338 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572322.195442:0:2383:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788572322.195443:0:2383:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788572322.195445:0:2383:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada456bb400. 02000000:00000001:0.0:1788572322.195446:0:2383:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788572322.195447:0:2383:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:0.0:1788572322.195448:0:2383:0:(genops.c:1223:class_import_put()) import ffff9ada58b08000 refcount=3 obd=lustre-MDT0001-mdc-ffff9ada58b0a000 00000020:00000001:0.0:1788572322.195450:0:2383:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:0.0:1788572322.195451:0:2383:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada48e6f400. 02000000:00000001:0.0:1788572322.195452:0:2383:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788572322.195464:0:2383:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788572322.195465:0:2383:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada456bea00. 00000100:00000001:0.0:1788572322.195468:0:2383:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788572322.195469:0:2383:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572322.195470:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572322.195477:0:2383:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788572322.195478:0:2383:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572322.195480:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788572322.195483:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:1.0:1788572322.201906:0:2384:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788572322.201916:0:2384:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788572322.201921:0:2384:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada600d1268. 00000400:00000200:1.0:1788572322.201934:0:2384:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d23b8 00000400:00000010:1.0:1788572322.201936:0:2384:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d23b8. 00000100:00000001:1.0:1788572322.201944:0:2384:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:1.0:1788572322.201958:0:2384:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada456bce00 x1875449616091264/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572338 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788572322.201970:0:2384:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada456bce00 x1875449616091264/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572338 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572322.201978:0:2384:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788572322.201980:0:2384:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572322.201983:0:2384:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1788572322.201987:0:2384:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788572322.201990:0:2384:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788572322.201992:0:2384:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572322.201995:0:2384:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788572322.202007:0:2384:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada51140460 time=17 v=5 (5 5 5 5) 00000100:00001000:1.0:1788572322.202011:0:2384:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada51140418 time=113 v=5 (2 0 0 0) 00000100:00000001:1.0:1788572322.202013:0:2384:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788572322.202015:0:2384:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572322.202019:0:2384:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788572322.202025:0:2384:0:(ldlm_request.c:1703:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572322.202029:0:2384:0:(client.c:3110:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788572322.202031:0:2384:0:(client.c:3122:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: skip recheck: last_committed 219043334508 00000100:00000001:1.0:1788572322.202033:0:2384:0:(client.c:3123:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788572322.202035:0:2384:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572322.202039:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada456bce00 x1875449616091264/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572338 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572322.202046:0:2384:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572322.202049:0:2384:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788572322.202051:0:2384:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788572322.202055:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada456bce00 x1875449616091264/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572338 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572322.202062:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada456bce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616091264:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572322.202067:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572322.202072:0:2384:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788572322.202075:0:2384:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada456bce00 x1875449616091264/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572338 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572322.202082:0:2384:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788572322.202085:0:2384:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788572322.202089:0:2384:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada607e1000. 02000000:00000001:1.0:1788572322.202094:0:2384:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788572322.202098:0:2384:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788572322.202100:0:2384:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=3 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:1.0:1788572322.202103:0:2384:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1788572322.202106:0:2384:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada48e6fa00. 02000000:00000001:1.0:1788572322.202110:0:2384:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788572322.202113:0:2384:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788572322.202115:0:2384:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada456bce00. 00000100:00000001:1.0:1788572322.202119:0:2384:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788572322.202127:0:2384:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572322.202130:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572322.202141:0:2384:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788572322.202143:0:2384:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572322.202147:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572322.202150:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572322.679000:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572322.679007:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572322.679014:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572322.679015:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572322.679019:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572322.679020:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572322.679024:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572322.679028:0:365:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:2.0:1788572322.679031:0:365:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:2.0:1788572322.679040:0:365:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:2.0:1788572322.679042:0:365:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:2.0:1788572322.679044:0:365:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572322.679048:0:365:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:2.0:1788572322.679050:0:365:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572322.679052:0:365:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572322.679055:0:365:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572322.679258:0:33423:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572322.679262:0:33423:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572322.679265:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572323.703183:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572323.703195:0:365:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:2.0:1788572323.703198:0:365:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:2.0:1788572323.703202:0:365:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572323.703208:0:365:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:2.0:1788572323.703211:0:365:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572323.703214:0:365:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572323.703216:0:365:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572323.703222:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572323.703223:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572323.703226:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572323.703227:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572323.703231:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572323.703232:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572323.713236:0:15205:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572323.713247:0:15205:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572323.713254:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572324.727075:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572324.727084:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572324.727091:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572324.727093:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572324.727096:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572324.727097:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572324.727101:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572324.727102:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572324.732818:0:15204:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572324.732829:0:15204:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572324.732835:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572325.751642:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572325.753826:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572325.753844:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572325.753848:0:365:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:2.0:1788572325.753850:0:365:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:2.0:1788572325.753876:0:365:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572325.753880:0:365:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:2.0:1788572325.753883:0:365:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572325.753885:0:365:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572325.753889:0:365:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572325.753893:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572325.753895:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572325.753899:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572325.753900:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788572325.754513:0:33423:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788572325.754517:0:33423:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788572325.754521:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572326.777561:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572326.777570:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572326.777577:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572326.777587:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572326.777591:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572326.777593:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572326.777597:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572326.777598:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572326.777838:0:15205:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572326.777842:0:15205:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572326.777845:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1788572327.287533:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788572327.287545:0:58077:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788572327.287559:0:58077:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada60625f80. 00000020:00000040:2.0:1788572327.287565:0:58077:0:(genops.c:1210:class_import_get()) import ffff9ada58a62800 refcount=4 obd=MGC192.168.202.113@tcp 00000100:00000001:2.0:1788572327.287570:0:58077:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788572327.287573:0:58077:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788572327.287580:0:58077:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788572327.287589:0:58077:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada617e8f00. 00000100:00000001:2.0:1788572327.287623:0:58077:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572327.287637:0:58077:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS req@ffff9ada60625f80 x1875449616091648/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788572327.287649:0:58077:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788572327.287653:0:58077:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572327.287659:0:58077:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000009212aa16] to pc [ptlrpcd_01_00+0] req@ffff9ada60625f80 x1875449616091648/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572327.287690:0:58077:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572327.287693:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788572327.287697:0:58077:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788572327.287700:0:58077:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada60627800. 00000020:00000040:2.0:1788572327.287703:0:58077:0:(genops.c:1210:class_import_get()) import ffff9ada51140000 refcount=4 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000100:00000001:2.0:1788572327.287706:0:58077:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788572327.287707:0:58077:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788572327.287709:0:58077:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788572327.287712:0:58077:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada617e8500. 00000100:00000001:2.0:1788572327.287715:0:58077:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572327.287719:0:58077:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID req@ffff9ada60627800 x1875449616091776/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788572327.287726:0:58077:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788572327.287728:0:58077:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572327.287731:0:58077:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000001b5935b9] to pc [ptlrpcd_01_01+1] req@ffff9ada60627800 x1875449616091776/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572327.287739:0:58077:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572327.287741:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788572327.287744:0:58077:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788572327.287746:0:58077:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada60624000. 00000020:00000040:2.0:1788572327.287748:0:58077:0:(genops.c:1210:class_import_get()) import ffff9ada58b08000 refcount=4 obd=lustre-MDT0001-mdc-ffff9ada58b0a000 00000100:00000001:2.0:1788572327.287750:0:58077:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788572327.287751:0:58077:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788572327.287752:0:58077:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788572327.287761:0:58077:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada509ed000. 00000100:00000001:2.0:1788572327.287764:0:58077:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572327.287767:0:58077:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID req@ffff9ada60624000 x1875449616091904/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788572327.287774:0:58077:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788572327.287776:0:58077:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572327.287779:0:58077:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000004fb47ed3] to pc [ptlrpcd_01_00+0] req@ffff9ada60624000 x1875449616091904/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572327.287785:0:58077:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00080000:2.0:1788572327.287787:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572327.287791:0:58077:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 00000100:00000001:2.0:1788572327.287794:0:58077:0:(import.c:745:ptlrpc_connect_import_locked()) Process entered 00000100:00080000:2.0:1788572327.287797:0:58077:0:(import.c:65:import_set_state_nolock()) ffff9ada48407800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00000001:2.0:1788572327.287801:0:58077:0:(import.c:520:import_select_connection()) Process entered 00000100:00080000:2.0:1788572327.287804:0:58077:0:(import.c:545:import_select_connection()) lustre-OST0000-osc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4239 00000400:00000200:2.0:1788572327.287812:0:58077:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572327.287820:0:58077:0:(peer.c:1689:LNetPeerDiscovered()) Peer NID 192.168.202.113@tcp is discovered: rc = 1 00000100:00080000:2.0:1788572327.287824:0:58077:0:(import.c:630:import_select_connection()) lustre-OST0000-osc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00000040:2.0:1788572327.287827:0:58077:0:(lustre_net.h:1977:ptlrpc_connection_put()) PUT conn=ffff9ada51db1200 refcount 14 to 192.168.202.113@tcp 00000100:00000001:2.0:1788572327.287831:0:58077:0:(connection.c:157:ptlrpc_connection_addref()) Process entered 00000100:00000040:2.0:1788572327.287833:0:58077:0:(connection.c:162:ptlrpc_connection_addref()) conn=ffff9ada51db1200 refcount 15 to 192.168.202.113@tcp 00000100:00000001:2.0:1788572327.287835:0:58077:0:(connection.c:164:ptlrpc_connection_addref()) Process leaving (rc=18446632861199700480 : -111212509851136 : ffff9ada51db1200) 00000020:00000001:2.0:1788572327.287838:0:58077:0:(genops.c:792:class_conn2export()) Process entered 00000020:00000040:2.0:1788572327.287849:0:58077:0:(genops.c:804:class_conn2export()) looking for export cookie 0xef30c340bfb39c83 00000020:00000001:2.0:1788572327.287876:0:58077:0:(lustre_handles.c:130:class_handle2object()) Process entered 00000020:00000040:2.0:1788572327.287880:0:58077:0:(lustre_handles.c:148:class_handle2object()) GET export ffff9ada50c3b800 refcount=4 00000020:00000001:2.0:1788572327.287882:0:58077:0:(lustre_handles.c:155:class_handle2object()) Process leaving (rc=18446632861181392896 : -111212528158720 : ffff9ada50c3b800) 00000020:00000001:2.0:1788572327.287885:0:58077:0:(genops.c:806:class_conn2export()) Process leaving (rc=18446632861181392896 : -111212528158720 : ffff9ada50c3b800) 00000100:00000040:2.0:1788572327.287889:0:58077:0:(lustre_net.h:1977:ptlrpc_connection_put()) PUT conn=ffff9ada51db1200 refcount 14 to 192.168.202.113@tcp 00000100:00000001:2.0:1788572327.287891:0:58077:0:(connection.c:157:ptlrpc_connection_addref()) Process entered 00000100:00000040:2.0:1788572327.287893:0:58077:0:(connection.c:162:ptlrpc_connection_addref()) conn=ffff9ada51db1200 refcount 15 to 192.168.202.113@tcp 00000100:00000001:2.0:1788572327.287894:0:58077:0:(connection.c:164:ptlrpc_connection_addref()) Process leaving (rc=18446632861199700480 : -111212509851136 : ffff9ada51db1200) 00000020:00000040:2.0:1788572327.287898:0:58077:0:(genops.c:879:class_export_put()) PUTting export ffff9ada50c3b800 : new refcount 3 00000100:00080000:2.0:1788572327.287902:0:58077:0:(import.c:665:import_select_connection()) lustre-OST0000-osc-ffff9ada58b0a000: import ffff9ada48407800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00000001:2.0:1788572327.287904:0:58077:0:(import.c:672:import_select_connection()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572327.287907:0:58077:0:(sec.c:1579:sptlrpc_import_sec_adapt()) Process entered 02000000:00000001:2.0:1788572327.287914:0:58077:0:(sec.c:1619:sptlrpc_import_sec_adapt()) Process leaving via out (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788572327.287917:0:58077:0:(sec.c:1647:sptlrpc_import_sec_adapt()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572327.287919:0:58077:0:(obd_class.h:888:obd_reconnect()) Process entered 00000008:00100000:2.0:1788572327.287924:0:58077:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683394 ocd_grant: 8388608, lost: 0. 00000008:00000001:2.0:1788572327.287927:0:58077:0:(osc_request.c:3833:osc_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572327.287930:0:58077:0:(obd_class.h:902:obd_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1788572327.287934:0:58077:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada60625c00. 00000020:00000040:2.0:1788572327.287936:0:58077:0:(genops.c:1210:class_import_get()) import ffff9ada48407800 refcount=3 obd=lustre-OST0000-osc-ffff9ada58b0a000 02000000:00000001:2.0:1788572327.287939:0:58077:0:(sec.c:2038:sptlrpc_sepol_get()) Process entered 02000000:00000001:2.0:1788572327.287941:0:58077:0:(sec.c:2048:sptlrpc_sepol_get()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572327.287943:0:58077:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788572327.287945:0:58077:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788572327.287947:0:58077:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788572327.287951:0:58077:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 1024 at ffff9ada60569800. 00000100:00000001:2.0:1788572327.287954:0:58077:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788572327.287959:0:58077:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616092032/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788572327.287967:0:58077:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788572327.287969:0:58077:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572327.287972:0:58077:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000abaebe17] to pc [ptlrpcd_rcv+-1] req@ffff9ada60625c00 x1875449616092032/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572327.287986:0:58077:0:(import.c:877:ptlrpc_connect_import_locked()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572327.287988:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:2.0:1788572327.287992:0:58077:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:2.0:1788572327.287997:0:58077:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (4249) 00000100:00000001:2.0:1788572327.288025:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788572327.288029:0:2382:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788572327.288031:0:2382:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788572327.288035:0:2382:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada60625c00 x1875449616092032/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572327.288054:0:2382:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788572327.288056:0:2382:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572327.288060:0:2382:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788572327.288062:0:2382:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788572327.288064:0:2382:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788572327.288068:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616092032:192.168.202.113@tcp:8:kworker.0 00000100:00000001:2.0:1788572327.288075:0:2382:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788572327.288080:0:2382:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788572327.288082:0:2382:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572327.288085:0:2382:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788572327.288090:0:2382:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at ffff9ada6056bc00. 02000000:00000001:2.0:1788572327.288092:0:2382:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788572327.288099:0:2382:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:2.0:1788572327.288103:0:2382:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada43d967e8. 00000400:00000010:2.0:1788572327.288110:0:2382:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada43c8fcc0. 00000100:00000200:2.0:1788572327.288118:0:2382:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1875449616092032, portal 4 00000100:00000001:2.0:1788572327.288121:0:2382:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788572327.288122:0:2382:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632861443447808 : -111212266103808 : ffff9ada60625c00) 00000100:00000040:2.0:1788572327.288130:0:2382:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada60625c00 x1875449616092032/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572352 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572327.288137:0:2382:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788572327.288140:0:2382:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:2.0:1788572327.288143:0:2382:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada43c8f990. 00000100:00000200:2.0:1788572327.288146:0:2382:0:(niobuf.c:94:ptl_send_buf()) Sending 520 bytes to portal 28, xid 1875449616092032, offset 0 00000400:00000200:2.0:1788572327.288152:0:2382:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:2.0:1788572327.288161:0:2382:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572327.288168:0:2382:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:2.0:1788572327.288174:0:2382:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788572327.288179:0:2382:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:2.0:1788572327.288183:0:2382:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788572327.288186:0:2382:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:2.0:1788572327.288191:0:2382:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22286:22286:256:4294967295] 192.168.202.113@tcp LPNI seq info [22282:22282:8:4294967295] 00000400:00000200:2.0:1788572327.288195:0:2382:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572327.288217:0:2382:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:2.0:1788572327.288224:0:2382:0:(socklnd_cb.c:931:ksocknal_send()) sending 520 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:2.0:1788572327.288228:0:2382:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada509ed400. 00000800:00000200:2.0:1788572327.288236:0:2382:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:2.0:1788572327.288244:0:2382:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:2.0:1788572327.288249:0:2382:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada509ed400 type 1, nob 616 niov 1 nkiov 1 00000100:00000001:2.0:1788572327.288258:0:2382:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572327.288263:0:2382:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: using a timeout of 18885 usecs (19 jiffies) 00000400:00000200:2.0:1788572327.288266:0:2382:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:2.0:1788572327.288269:0:2382:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298912276 00000100:00000001:2.0:1788572327.288272:0:2382:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298917709 00000100:00000001:2.0:1788572327.288274:0:2382:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298912276 00000100:00000001:2.0:1788572327.288276:0:2382:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298917709 00000100:00000001:2.0:1788572327.288278:0:2382:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788572327.288281:0:2382:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572327.288284:0:2382:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572327.288287:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572327.288294:0:2382:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788572327.288296:0:2382:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=5 : 5 : 5) 00000100:00000001:2.0:1788572327.288297:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788572327.288298:0:2382:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788572327.288300:0:2382:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572327.288302:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572327.288978:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788572327.288983:0:2385:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788572327.288985:0:2385:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788572327.288995:0:2385:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada60625f80 x1875449616091648/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572327.289005:0:2385:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788572327.289007:0:2385:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788572327.289010:0:2385:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788572327.289012:0:2385:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788572327.289014:0:2385:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788572327.289017:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449616091648:192.168.202.113@tcp:400:kworker.0 00000100:00000001:3.0:1788572327.289021:0:2385:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788572327.289026:0:2385:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788572327.289028:0:2385:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788572327.289030:0:2385:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788572327.289036:0:2385:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada50a3b200. 02000000:00000001:3.0:1788572327.289038:0:2385:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788572327.289104:0:2385:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:3.0:1788572327.289109:0:2385:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada510c9f78. 00000400:00000010:3.0:1788572327.289117:0:2385:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada512695d8. 00000100:00000200:3.0:1788572327.289121:0:2385:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616091648, portal 25 00000100:00000001:3.0:1788572327.289123:0:2385:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788572327.289124:0:2385:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632861443448704 : -111212266102912 : ffff9ada60625f80) 00000100:00000040:3.0:1788572327.289131:0:2385:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada60625f80 x1875449616091648/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572343 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572327.289148:0:2385:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788572327.289151:0:2385:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:3.0:1788572327.289154:0:2385:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada51269330. 00000100:00000200:3.0:1788572327.289158:0:2385:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875449616091648, offset 0 00000400:00000200:3.0:1788572327.289163:0:2385:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:3.0:1788572327.289168:0:2385:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572327.289173:0:2385:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:3.0:1788572327.289177:0:2385:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788572327.289181:0:2385:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:3.0:1788572327.289184:0:2385:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788572327.289187:0:2385:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:3.0:1788572327.289190:0:2385:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22287:22287:255:4294967295] 192.168.202.113@tcp LPNI seq info [22283:22283:7:4294967295] 00000400:00000200:3.0:1788572327.289195:0:2385:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572327.289320:0:2385:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:3.0:1788572327.289327:0:2385:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:3.0:1788572327.289333:0:2385:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada50c31700. 00000800:00000200:3.0:1788572327.289339:0:2385:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:3.0:1788572327.289345:0:2385:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:3.0:1788572327.289349:0:2385:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada50c31700 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788572327.289353:0:2385:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788572327.289357:0:2385:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:3.0:1788572327.289359:0:2385:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS new deadline estimation for cpu 2 is 4298922830 00000100:00000001:3.0:1788572327.289366:0:2385:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4298917709 00000100:00000001:3.0:1788572327.289371:0:2385:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS new deadline estimation for cpu 3 is 4298922830 00000100:00000001:3.0:1788572327.289375:0:2385:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4298917709 00000100:00000001:3.0:1788572327.289379:0:2385:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788572327.289382:0:2385:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572327.289384:0:2385:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788572327.289390:0:2385:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada60624000 x1875449616091904/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572327.289397:0:2385:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788572327.289399:0:2385:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788572327.289402:0:2385:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788572327.289404:0:2385:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788572327.289406:0:2385:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788572327.289409:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616091904:192.168.202.113@tcp:400:kworker.0 00000100:00000001:3.0:1788572327.289412:0:2385:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788572327.289415:0:2385:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788572327.289417:0:2385:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788572327.289419:0:2385:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788572327.289422:0:2385:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada50a3a600. 02000000:00000001:3.0:1788572327.289424:0:2385:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788572327.289427:0:2385:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:3.0:1788572327.289430:0:2385:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada510c9898. 00000400:00000010:3.0:1788572327.289433:0:2385:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada51269ee0. 00000100:00000200:3.0:1788572327.289436:0:2385:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616091904, portal 10 00000100:00000001:3.0:1788572327.289438:0:2385:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788572327.289439:0:2385:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632861443440640 : -111212266110976 : ffff9ada60624000) 00000100:00000040:3.0:1788572327.289444:0:2385:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada60624000 x1875449616091904/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572343 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572327.289451:0:2385:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788572327.289453:0:2385:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:3.0:1788572327.289456:0:2385:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada51269110. 00000100:00000200:3.0:1788572327.289458:0:2385:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875449616091904, offset 0 00000400:00000200:3.0:1788572327.289462:0:2385:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:3.0:1788572327.289465:0:2385:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572327.289468:0:2385:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:3.0:1788572327.289471:0:2385:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788572327.289474:0:2385:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:3.0:1788572327.289476:0:2385:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788572327.289478:0:2385:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:3.0:1788572327.289481:0:2385:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22288:22288:254:4294967295] 192.168.202.113@tcp LPNI seq info [22284:22284:6:4294967295] 00000400:00000200:3.0:1788572327.289486:0:2385:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572327.289492:0:2385:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:3.0:1788572327.289496:0:2385:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:3.0:1788572327.289499:0:2385:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada50c31400. 00000800:00000200:3.0:1788572327.289502:0:2385:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:3.0:1788572327.289506:0:2385:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:3.0:1788572327.289509:0:2385:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada50c31400 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788572327.289511:0:2385:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572327.289514:0:2385:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: using a timeout of 16608 usecs (17 jiffies) 00000400:00000200:3.0:1788572327.289517:0:2385:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:3.0:1788572327.289519:0:2385:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298912275 00000100:00000001:3.0:1788572327.289521:0:2385:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298922830 00000100:00000001:3.0:1788572327.289523:0:2385:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298912275 00000100:00000001:3.0:1788572327.289525:0:2385:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298922830 00000100:00000001:3.0:1788572327.289527:0:2385:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788572327.289529:0:2385:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572327.289531:0:2385:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572327.289534:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788572327.289541:0:2385:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788572327.289542:0:2385:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788572327.289545:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788572327.289546:0:2385:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788572327.289547:0:2385:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788572327.289550:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00000001:3.0:1788572327.289552:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788572327.289554:0:2385:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788572327.289555:0:2385:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788572327.289557:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788572327.289558:0:2385:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788572327.289559:0:2385:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788572327.289565:0:2385:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada60627800 x1875449616091776/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572327.289572:0:2385:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788572327.289574:0:2385:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788572327.289576:0:2385:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788572327.289577:0:2385:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788572327.289579:0:2385:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788572327.289582:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616091776:192.168.202.113@tcp:400:kworker.0 00000100:00000001:3.0:1788572327.289586:0:2385:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788572327.289588:0:2385:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788572327.289589:0:2385:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788572327.289591:0:2385:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788572327.289604:0:2385:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada50a3aa00. 02000000:00000001:3.0:1788572327.289606:0:2385:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788572327.289609:0:2385:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:3.0:1788572327.289612:0:2385:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada510c9b00. 00000400:00000010:3.0:1788572327.289615:0:2385:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada51269b28. 00000100:00000200:3.0:1788572327.289617:0:2385:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616091776, portal 10 00000100:00000001:3.0:1788572327.289619:0:2385:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788572327.289621:0:2385:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632861443454976 : -111212266096640 : ffff9ada60627800) 00000100:00000040:3.0:1788572327.289627:0:2385:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada60627800 x1875449616091776/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572343 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572327.289633:0:2385:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788572327.289636:0:2385:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:3.0:1788572327.289638:0:2385:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada51269f68. 00000100:00000200:3.0:1788572327.289640:0:2385:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875449616091776, offset 0 00000400:00000200:3.0:1788572327.289644:0:2385:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:3.0:1788572327.289647:0:2385:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572327.289650:0:2385:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:3.0:1788572327.289652:0:2385:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788572327.289655:0:2385:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:3.0:1788572327.289657:0:2385:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788572327.289660:0:2385:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:3.0:1788572327.289663:0:2385:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22289:22289:253:4294967295] 192.168.202.113@tcp LPNI seq info [22285:22285:5:4294967295] 00000400:00000200:3.0:1788572327.289667:0:2385:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572327.289673:0:2385:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:3.0:1788572327.289677:0:2385:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:3.0:1788572327.289679:0:2385:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada50c31a00. 00000800:00000200:3.0:1788572327.289683:0:2385:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:3.0:1788572327.289687:0:2385:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:3.0:1788572327.289690:0:2385:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada50c31a00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788572327.289692:0:2385:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572327.289695:0:2385:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: using a timeout of 41270 usecs (42 jiffies) 00000400:00000200:3.0:1788572327.289698:0:2385:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:3.0:1788572327.289699:0:2385:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298912300 00000100:00000001:3.0:1788572327.289702:0:2385:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298922830 00000100:00000001:3.0:1788572327.289704:0:2385:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298912300 00000100:00000001:3.0:1788572327.289706:0:2385:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298922830 00000100:00000001:3.0:1788572327.289707:0:2385:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788572327.289709:0:2385:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572327.289711:0:2385:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572327.289713:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572327.289727:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788572327.289729:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788572327.289776:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788572327.289779:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada509ed400. 00000400:00000200:3.0:1788572327.289789:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572327.289795:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:3.0:1788572327.289801:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada43c8f990 00000400:00000010:3.0:1788572327.289804:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada43c8f990. 00000100:00000001:3.0:1788572327.289809:0:2377:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:3.0:1788572327.289815:0:2377:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada60625c00 x1875449616092032/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572352 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572327.289829:0:2377:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572327.289833:0:2377:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada60625c00 x1875449616092032/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572352 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572327.291184:0:2377:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572327.291192:0:2377:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000200:3.0:1788572327.291992:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788572327.292000:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada50c31700. 00000400:00000200:3.0:1788572327.292010:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572327.292021:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:3.0:1788572327.292029:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada51269330 00000400:00000010:3.0:1788572327.292032:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada51269330. 00000100:00000001:3.0:1788572327.292037:0:2377:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:3.0:1788572327.292334:0:2377:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada60625f80 x1875449616091648/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572343 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572327.292348:0:2377:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572327.292352:0:2377:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada60625f80 x1875449616091648/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572343 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572327.292357:0:2377:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572327.292359:0:2377:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000200:3.0:1788572327.292376:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788572327.292379:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada50c31400. 00000400:00000200:3.0:1788572327.292382:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572327.292385:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:3.0:1788572327.292388:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada51269110 00000400:00000010:3.0:1788572327.292390:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada51269110. 00000100:00000001:3.0:1788572327.292393:0:2377:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:3.0:1788572327.292397:0:2377:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada60624000 x1875449616091904/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572343 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572327.292404:0:2377:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572327.292407:0:2377:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada60624000 x1875449616091904/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572343 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572327.292421:0:2377:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572327.292422:0:2377:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000200:3.0:1788572327.292512:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788572327.292515:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada50c31a00. 00000400:00000200:3.0:1788572327.292518:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572327.292522:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:3.0:1788572327.292524:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada51269f68 00000400:00000010:3.0:1788572327.292526:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada51269f68. 00000100:00000001:3.0:1788572327.292529:0:2377:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:3.0:1788572327.292534:0:2377:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada60627800 x1875449616091776/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572343 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572327.292541:0:2377:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572327.292544:0:2377:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada60627800 x1875449616091776/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572343 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572327.292549:0:2377:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572327.292550:0:2377:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000001:3.0:1788572327.294956:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572327.294974:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572327.294979:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572327.294982:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572327.294992:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572327.295002:0:2378:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572327.295012:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572327.295017:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572327.295024:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572327.295029:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 25 MB=0x6a9b624294a00 00000400:00000200:3.0:1788572327.295035:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572327.295041:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.202.113@tcp of length 224/224 into md 0x35755 [1] + 224 00000800:00000001:3.0:1788572327.295050:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572327.295054:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572327.295058:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572327.295063:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572327.295074:0:2378:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572327.295084:0:2378:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada60625f80 x1875449616091648/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572343 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572327.295096:0:2378:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada60625f80 x1875449616091648/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572343 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572327.295123:0:2378:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572327.295133:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572327.295137:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572327.295139:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572327.295141:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572327.295145:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572327.295148:0:2378:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572327.295152:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572327.295155:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572327.295157:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572327.295160:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 4 MB=0x6a9b624294b80 00000400:00000200:3.0:1788572327.295163:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572327.295168:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.202.113@tcp of length 224/224 into md 0x35745 [1] + 0 00000800:00000001:3.0:1788572327.295172:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572327.295237:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572327.295240:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572327.295243:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572327.295247:0:2378:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572327.295252:0:2378:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada60625c00 x1875449616092032/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572352 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572327.295260:0:2378:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=0 replen=544 req@ffff9ada60625c00 x1875449616092032/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572352 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572327.295283:0:2378:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572327.295288:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572327.295291:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572327.295334:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788572327.295338:0:2382:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788572327.295346:0:2382:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788572327.295348:0:2382:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788572327.295352:0:2382:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada43d967e8. 00000400:00000200:0.0:1788572327.295358:0:2382:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada43c8fcc0 00000400:00000010:0.0:1788572327.295360:0:2382:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada43c8fcc0. 00000100:00000001:0.0:1788572327.295364:0:2382:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:0.0:1788572327.295369:0:2382:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada60625c00 x1875449616092032/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572352 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788572327.295378:0:2382:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada60625c00 x1875449616092032/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572352 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572327.295384:0:2382:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788572327.295386:0:2382:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572327.295389:0:2382:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:0.0:1788572327.295393:0:2382:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788572327.295396:0:2382:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788572327.295398:0:2382:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572327.295400:0:2382:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788572327.295411:0:2382:0:(import.c:2195:obd_at_measure()) add 0 to ffff9ada48407c60 time=172 v=5 (5 5 5 5) 00000100:00001000:0.0:1788572327.295416:0:2382:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada48407c18 time=49 v=20 (20 1 1 1) 00000100:00000001:0.0:1788572327.295419:0:2382:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788572327.295422:0:2382:0:(client.c:1506:ptlrpc_check_status()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000001:0.0:1788572327.295425:0:2382:0:(client.c:1712:after_reply()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000040:0.0:1788572327.295430:0:2382:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada60625c00 x1875449616092032/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/224 e 0 to 0 dl 1788572352 ref 1 fl Rpc:RNQU/200/0 rc -19/-19 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572327.295437:0:2382:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=18446744073709551597 : -19 : 0xffffffffffffffed) 00000100:00000001:0.0:1788572327.295441:0:2382:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788572327.295443:0:2382:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572327.295446:0:2382:0:(import.c:1101:ptlrpc_connect_interpret()) Process entered 00000100:00000001:0.0:1788572327.295448:0:2382:0:(import.c:1136:ptlrpc_connect_interpret()) Process leaving via out (rc=18446744073709551597 : -19 : 0xffffffffffffffed) 00000100:00080000:0.0:1788572327.295452:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada48407800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788572327.295458:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.113@tcp failed (-19) 00000100:00000001:0.0:1788572327.295461:0:2382:0:(import.c:1561:ptlrpc_connect_interpret()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000040:0.0:1788572327.295466:0:2382:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada60625c00 x1875449616092032/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/224 e 0 to 0 dl 1788572352 ref 1 fl Interpret:RNQU/200/0 rc -19/-19 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572327.295474:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616092032:192.168.202.113@tcp:8:kworker.0 00000100:00000001:0.0:1788572327.295479:0:2382:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572327.295483:0:2382:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788572327.295486:0:2382:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60625c00 x1875449616092032/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/224 e 0 to 0 dl 1788572352 ref 1 fl Complete:RNQU/200/0 rc -19/-19 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572327.295492:0:2382:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788572327.295495:0:2382:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788572327.295499:0:2382:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada6056bc00. 02000000:00000001:0.0:1788572327.295502:0:2382:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788572327.295505:0:2382:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:0.0:1788572327.295507:0:2382:0:(genops.c:1223:class_import_put()) import ffff9ada48407800 refcount=2 obd=lustre-OST0000-osc-ffff9ada58b0a000 00000020:00000001:0.0:1788572327.295510:0:2382:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:0.0:1788572327.295513:0:2382:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 1024 at ffff9ada60569800. 02000000:00000001:0.0:1788572327.295516:0:2382:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788572327.295518:0:2382:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788572327.295520:0:2382:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60625c00. 00000100:00000001:0.0:1788572327.295524:0:2382:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788572327.295526:0:2382:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572327.295529:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572327.295537:0:2382:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788572327.295539:0:2382:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572327.295541:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788572327.295543:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572327.295610:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788572327.295613:0:2385:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788572327.295619:0:2385:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788572327.295620:0:2385:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788572327.295622:0:2385:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada510c9f78. 00000400:00000200:2.0:1788572327.295627:0:2385:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada512695d8 00000400:00000010:2.0:1788572327.295629:0:2385:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada512695d8. 00000100:00000001:2.0:1788572327.295631:0:2385:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:2.0:1788572327.295636:0:2385:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada60625f80 x1875449616091648/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572343 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788572327.295644:0:2385:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada60625f80 x1875449616091648/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572343 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572327.295649:0:2385:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788572327.295651:0:2385:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572327.295653:0:2385:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:2.0:1788572327.295655:0:2385:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788572327.295656:0:2385:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788572327.295657:0:2385:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572327.295659:0:2385:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788572327.295662:0:2385:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada58a62c60 time=89 v=5 (5 5 5 5) 00000100:00001000:2.0:1788572327.295665:0:2385:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada58a62c18 time=119 v=5 (1 0 0 0) 00000100:00000001:2.0:1788572327.295667:0:2385:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788572327.295669:0:2385:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572327.295673:0:2385:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788572327.295675:0:2385:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572327.295678:0:2385:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572327.295682:0:2385:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada60625f80 x1875449616091648/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572343 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572327.295688:0:2385:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788572327.295690:0:2385:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788572327.295691:0:2385:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572327.295695:0:2385:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada60625f80 x1875449616091648/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572343 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788572327.295702:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449616091648:192.168.202.113@tcp:400:kworker.0 00000100:00000001:2.0:1788572327.295707:0:2385:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572327.295709:0:2385:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572327.295712:0:2385:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60625f80 x1875449616091648/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572343 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572327.295717:0:2385:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572327.295719:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572327.295721:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada50a3b200. 02000000:00000001:2.0:1788572327.295724:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572327.295726:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572327.295727:0:2385:0:(genops.c:1223:class_import_put()) import ffff9ada58a62800 refcount=3 obd=MGC192.168.202.113@tcp 00000020:00000001:2.0:1788572327.295729:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572327.295731:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada617e8f00. 02000000:00000001:2.0:1788572327.295735:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572327.295737:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572327.295738:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60625f80. 00000100:00000001:2.0:1788572327.295741:0:2385:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572327.295742:0:2385:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572327.295745:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572327.295747:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788572327.295748:0:2385:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788572327.295749:0:2385:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572327.295751:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788572327.296736:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572327.296748:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572327.296752:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572327.296754:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572327.296762:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572327.296768:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572327.296777:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572327.296781:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572327.296786:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572327.296790:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 10 MB=0x6a9b624294b00 00000400:00000200:3.0:1788572327.296796:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572327.296812:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.113@tcp of length 224/224 into md 0x35765 [1] + 224 00000800:00000001:3.0:1788572327.296819:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572327.296827:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572327.296831:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572327.296836:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572327.296848:0:2377:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572327.296872:0:2377:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada60624000 x1875449616091904/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572343 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572327.296882:0:2377:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada60624000 x1875449616091904/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572343 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572327.296900:0:2377:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572327.296907:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572327.296910:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572327.297933:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572327.297943:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572327.297947:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572327.297948:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572327.297956:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572327.297962:0:2378:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572327.297970:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572327.297974:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572327.297979:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572327.297983:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 10 MB=0x6a9b624294a80 00000400:00000200:3.0:1788572327.297989:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572327.297994:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.113@tcp of length 224/224 into md 0x35775 [1] + 224 00000800:00000001:3.0:1788572327.298000:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572327.298057:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572327.298061:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572327.298066:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572327.298073:0:2378:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572327.298081:0:2378:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada60627800 x1875449616091776/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572343 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572327.298092:0:2378:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada60627800 x1875449616091776/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572343 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572327.298101:0:2378:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572327.298108:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572327.298111:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572327.298191:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788572327.298196:0:2385:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788572327.298213:0:2385:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788572327.298216:0:2385:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788572327.298219:0:2385:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada510c9b00. 00000400:00000200:2.0:1788572327.298226:0:2385:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada51269b28 00000400:00000010:2.0:1788572327.298228:0:2385:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada51269b28. 00000100:00000001:2.0:1788572327.298232:0:2385:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:2.0:1788572327.298237:0:2385:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada60627800 x1875449616091776/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572343 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788572327.298247:0:2385:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada60627800 x1875449616091776/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572343 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572327.298255:0:2385:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788572327.298257:0:2385:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572327.298259:0:2385:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:2.0:1788572327.298263:0:2385:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788572327.298266:0:2385:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788572327.298268:0:2385:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572327.298270:0:2385:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788572327.298280:0:2385:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada51140460 time=22 v=5 (5 5 5 5) 00000100:00001000:2.0:1788572327.298285:0:2385:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada51140418 time=118 v=5 (2 0 0 0) 00000100:00000001:2.0:1788572327.298288:0:2385:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788572327.298290:0:2385:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572327.298293:0:2385:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788572327.298298:0:2385:0:(ldlm_request.c:1703:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572327.298302:0:2385:0:(client.c:3110:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788572327.298304:0:2385:0:(client.c:3122:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: skip recheck: last_committed 219043334508 00000100:00000001:2.0:1788572327.298306:0:2385:0:(client.c:3123:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788572327.298309:0:2385:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572327.298314:0:2385:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada60627800 x1875449616091776/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572343 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572327.298322:0:2385:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788572327.298325:0:2385:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788572327.298327:0:2385:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572327.298331:0:2385:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada60627800 x1875449616091776/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572343 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788572327.298338:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616091776:192.168.202.113@tcp:400:kworker.0 00000400:00000001:2.0:1788572327.298345:0:2385:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788572327.298347:0:2385:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788572327.298349:0:2385:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada510c9898. 00000400:00000200:2.0:1788572327.298351:0:2385:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada51269ee0 00000400:00000010:2.0:1788572327.298353:0:2385:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada51269ee0. 00000100:00000001:2.0:1788572327.298355:0:2385:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:2.0:1788572327.298360:0:2385:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada60624000 x1875449616091904/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572343 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788572327.298368:0:2385:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada60624000 x1875449616091904/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572343 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572327.298374:0:2385:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788572327.298375:0:2385:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572327.298377:0:2385:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:2.0:1788572327.298379:0:2385:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788572327.298380:0:2385:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788572327.298381:0:2385:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572327.298383:0:2385:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788572327.298387:0:2385:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada58b08460 time=89 v=5 (5 5 5 5) 00000100:00001000:2.0:1788572327.298390:0:2385:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada58b08418 time=89 v=5 (1 1 1 1) 00000100:00000001:2.0:1788572327.298392:0:2385:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788572327.298393:0:2385:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572327.298394:0:2385:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788572327.298397:0:2385:0:(ldlm_request.c:1703:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572327.298400:0:2385:0:(client.c:3110:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788572327.298402:0:2385:0:(client.c:3122:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff9ada58b0a000: skip recheck: last_committed 0 00000100:00000001:2.0:1788572327.298404:0:2385:0:(client.c:3123:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788572327.298405:0:2385:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572327.298409:0:2385:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada60624000 x1875449616091904/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572343 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572327.298415:0:2385:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788572327.298417:0:2385:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788572327.298418:0:2385:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572327.298421:0:2385:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada60624000 x1875449616091904/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572343 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788572327.298427:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616091904:192.168.202.113@tcp:400:kworker.0 00000100:00000001:2.0:1788572327.298431:0:2385:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572327.298434:0:2385:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572327.298437:0:2385:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60627800 x1875449616091776/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572343 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572327.298443:0:2385:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572327.298445:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572327.298448:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada50a3aa00. 02000000:00000001:2.0:1788572327.298453:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572327.298455:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572327.298457:0:2385:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=3 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572327.298460:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572327.298462:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada617e8500. 02000000:00000001:2.0:1788572327.298466:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572327.298468:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572327.298470:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60627800. 00000100:00000001:2.0:1788572327.298474:0:2385:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572327.298475:0:2385:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572327.298477:0:2385:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572327.298481:0:2385:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada60624000 x1875449616091904/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572343 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572327.298492:0:2385:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572327.298494:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572327.298495:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada50a3a600. 02000000:00000001:2.0:1788572327.298497:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572327.298499:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572327.298500:0:2385:0:(genops.c:1223:class_import_put()) import ffff9ada58b08000 refcount=3 obd=lustre-MDT0001-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572327.298502:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572327.298504:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada509ed000. 02000000:00000001:2.0:1788572327.298507:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572327.298508:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572327.298509:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada60624000. 00000100:00000001:2.0:1788572327.298512:0:2385:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572327.298513:0:2385:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572327.298516:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572327.298525:0:2385:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788572327.298527:0:2385:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572327.298529:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788572327.298532:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572327.799008:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572327.799018:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572327.799025:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572327.799026:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572327.799030:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572327.799031:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572327.799035:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572327.799036:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572327.799323:0:15204:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572327.799327:0:15204:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572327.799330:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572328.825427:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572328.825436:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572328.825446:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572328.825447:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572328.825452:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572328.825453:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572328.825457:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572328.825459:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788572328.828306:0:33423:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788572328.828317:0:33423:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788572328.828323:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572329.847018:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572329.847025:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572329.847033:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572329.847034:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572329.847037:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572329.847038:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572329.847041:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572329.847042:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572329.850423:0:15205:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572329.850430:0:15205:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572329.850435:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572330.872089:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572330.872101:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572330.872111:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572330.872112:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572330.872117:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572330.872118:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572330.872122:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572330.872123:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572330.872265:0:15204:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572330.872269:0:15204:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572330.872273:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572331.895217:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572331.895227:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572331.895234:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572331.895236:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572331.895241:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572331.895245:0:365:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:2.0:1788572331.895248:0:365:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:2.0:1788572331.895257:0:365:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:2.0:1788572331.895260:0:365:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:2.0:1788572331.895262:0:365:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572331.895266:0:365:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:2.0:1788572331.895269:0:365:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572331.895271:0:365:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572331.895274:0:365:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572331.895278:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572331.895279:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788572331.895437:0:33423:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788572331.895442:0:33423:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788572331.895446:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788572332.408436:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788572332.408448:0:58077:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788572332.408463:0:58077:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada456bce00. 00000020:00000040:1.0:1788572332.408472:0:58077:0:(genops.c:1210:class_import_get()) import ffff9ada58a62800 refcount=4 obd=MGC192.168.202.113@tcp 00000100:00000001:1.0:1788572332.408477:0:58077:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788572332.408481:0:58077:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788572332.408488:0:58077:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788572332.408498:0:58077:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada48e6fa00. 00000100:00000001:1.0:1788572332.408507:0:58077:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572332.408523:0:58077:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS req@ffff9ada456bce00 x1875449616092160/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788572332.408535:0:58077:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788572332.408541:0:58077:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572332.408548:0:58077:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000000e2f9a7e] to pc [ptlrpcd_00_01+1] req@ffff9ada456bce00 x1875449616092160/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572332.408582:0:58077:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572332.408585:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788572332.408589:0:58077:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788572332.408593:0:58077:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada456bf800. 00000020:00000040:1.0:1788572332.408595:0:58077:0:(genops.c:1210:class_import_get()) import ffff9ada51140000 refcount=4 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000100:00000001:1.0:1788572332.408597:0:58077:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788572332.408598:0:58077:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788572332.408600:0:58077:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788572332.408606:0:58077:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada48e6f400. 00000100:00000001:1.0:1788572332.408611:0:58077:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572332.408615:0:58077:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID req@ffff9ada456bf800 x1875449616092288/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788572332.408621:0:58077:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788572332.408623:0:58077:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572332.408627:0:58077:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000000c5d7dd3] to pc [ptlrpcd_00_00+0] req@ffff9ada456bf800 x1875449616092288/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572332.408637:0:58077:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572332.408639:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788572332.408644:0:58077:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788572332.408646:0:58077:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada456bc000. 00000100:00000001:0.0:1788572332.408648:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000020:00000040:1.0:1788572332.408650:0:58077:0:(genops.c:1210:class_import_get()) import ffff9ada58b08000 refcount=4 obd=lustre-MDT0001-mdc-ffff9ada58b0a000 00000100:00000001:0.0:1788572332.408654:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788572332.408654:0:58077:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788572332.408656:0:58077:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:0.0:1788572332.408659:0:2384:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 02000000:00000001:1.0:1788572332.408660:0:58077:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788572332.408665:0:58077:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada48e6f100. 00000100:00000040:0.0:1788572332.408669:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada456bce00 x1875449616092160/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572332.408670:0:58077:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572332.408678:0:58077:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID req@ffff9ada456bc000 x1875449616092416/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788572332.408684:0:2384:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788572332.408687:0:2384:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572332.408690:0:58077:0:(jobid.c:959:lustre_get_jobid()) Process entered 02000000:00000001:0.0:1788572332.408694:0:2384:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:1.0:1788572332.408694:0:58077:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572332.408698:0:2384:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:1.0:1788572332.408701:0:58077:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002c2d0e3b] to pc [ptlrpcd_00_01+1] req@ffff9ada456bc000 x1875449616092416/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788572332.408703:0:2384:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788572332.408710:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada456bce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449616092160:192.168.202.113@tcp:400:kworker.0 00000100:00000001:0.0:1788572332.408719:0:2384:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:1.0:1788572332.408720:0:58077:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00080000:1.0:1788572332.408724:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 02000000:00000001:0.0:1788572332.408727:0:2384:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 00000100:00080000:1.0:1788572332.408730:0:58077:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-OST0000_UUID: starting recovery 02000000:00000001:0.0:1788572332.408744:0:2384:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572332.408748:0:2384:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788572332.408754:0:2384:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada456bba00. 02000000:00000001:0.0:1788572332.408756:0:2384:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788572332.408766:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:0.0:1788572332.408772:0:2384:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada60acacb8. 00000400:00000010:0.0:1788572332.408780:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada5176a770. 00000100:00000200:0.0:1788572332.408789:0:2384:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616092160, portal 25 00000100:00000001:0.0:1788572332.408792:0:2384:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788572332.408794:0:2384:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860991081984 : -111212718469632 : ffff9ada456bce00) 00000100:00000040:0.0:1788572332.408811:0:2384:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada456bce00 x1875449616092160/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572348 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572332.408819:0:2384:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788572332.408822:0:2384:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:0.0:1788572332.408825:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada5176a088. 00000100:00000200:0.0:1788572332.408830:0:2384:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875449616092160, offset 0 00000400:00000200:0.0:1788572332.408839:0:2384:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000100:00000001:1.0:1788572332.408841:0:58077:0:(import.c:745:ptlrpc_connect_import_locked()) Process entered 00000100:00080000:1.0:1788572332.408845:0:58077:0:(import.c:65:import_set_state_nolock()) ffff9ada48407800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00000001:1.0:1788572332.408893:0:58077:0:(import.c:520:import_select_connection()) Process entered 00000400:00000200:0.0:1788572332.408898:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00080000:1.0:1788572332.408899:0:58077:0:(import.c:545:import_select_connection()) lustre-OST0000-osc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4244 00000400:00000200:0.0:1788572332.408913:0:2384:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:0.0:1788572332.408920:0:2384:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788572332.408925:0:2384:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:0.0:1788572332.408931:0:2384:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788572332.408934:0:2384:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:0.0:1788572332.408939:0:2384:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22290:22290:256:4294967295] 192.168.202.113@tcp LPNI seq info [22286:22286:8:4294967295] 00000400:00000200:0.0:1788572332.408944:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572332.408957:0:58077:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:0.0:1788572332.408961:0:2384:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000400:00000200:1.0:1788572332.408967:0:58077:0:(peer.c:1689:LNetPeerDiscovered()) Peer NID 192.168.202.113@tcp is discovered: rc = 1 00000800:00000200:0.0:1788572332.408972:0:2384:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000100:00080000:1.0:1788572332.408972:0:58077:0:(import.c:630:import_select_connection()) lustre-OST0000-osc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00000040:1.0:1788572332.408979:0:58077:0:(lustre_net.h:1977:ptlrpc_connection_put()) PUT conn=ffff9ada51db1200 refcount 14 to 192.168.202.113@tcp 00000100:00000001:1.0:1788572332.408982:0:58077:0:(connection.c:157:ptlrpc_connection_addref()) Process entered 00000100:00000040:1.0:1788572332.408985:0:58077:0:(connection.c:162:ptlrpc_connection_addref()) conn=ffff9ada51db1200 refcount 15 to 192.168.202.113@tcp 00000800:00000010:0.0:1788572332.408987:0:2384:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada5074a400. 00000100:00000001:1.0:1788572332.408988:0:58077:0:(connection.c:164:ptlrpc_connection_addref()) Process leaving (rc=18446632861199700480 : -111212509851136 : ffff9ada51db1200) 00000020:00000001:1.0:1788572332.408994:0:58077:0:(genops.c:792:class_conn2export()) Process entered 00000020:00000040:1.0:1788572332.408997:0:58077:0:(genops.c:804:class_conn2export()) looking for export cookie 0xef30c340bfb39c83 00000800:00000200:0.0:1788572332.408999:0:2384:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000020:00000001:1.0:1788572332.409001:0:58077:0:(lustre_handles.c:130:class_handle2object()) Process entered 00000020:00000040:1.0:1788572332.409006:0:58077:0:(lustre_handles.c:148:class_handle2object()) GET export ffff9ada50c3b800 refcount=4 00000800:00000200:0.0:1788572332.409011:0:2384:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000020:00000001:1.0:1788572332.409011:0:58077:0:(lustre_handles.c:155:class_handle2object()) Process leaving (rc=18446632861181392896 : -111212528158720 : ffff9ada50c3b800) 00000020:00000001:1.0:1788572332.409016:0:58077:0:(genops.c:806:class_conn2export()) Process leaving (rc=18446632861181392896 : -111212528158720 : ffff9ada50c3b800) 00000800:00000200:0.0:1788572332.409019:0:2384:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada5074a400 type 1, nob 320 niov 1 nkiov 1 00000100:00000040:1.0:1788572332.409027:0:58077:0:(lustre_net.h:1977:ptlrpc_connection_put()) PUT conn=ffff9ada51db1200 refcount 14 to 192.168.202.113@tcp 00000100:00000001:1.0:1788572332.409032:0:58077:0:(connection.c:157:ptlrpc_connection_addref()) Process entered 00000100:00000040:1.0:1788572332.409035:0:58077:0:(connection.c:162:ptlrpc_connection_addref()) conn=ffff9ada51db1200 refcount 15 to 192.168.202.113@tcp 00000100:00000001:1.0:1788572332.409037:0:58077:0:(connection.c:164:ptlrpc_connection_addref()) Process leaving (rc=18446632861199700480 : -111212509851136 : ffff9ada51db1200) 00000020:00000040:1.0:1788572332.409043:0:58077:0:(genops.c:879:class_export_put()) PUTting export ffff9ada50c3b800 : new refcount 3 00000100:00000001:0.0:1788572332.409045:0:2384:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00080000:1.0:1788572332.409048:0:58077:0:(import.c:665:import_select_connection()) lustre-OST0000-osc-ffff9ada58b0a000: import ffff9ada48407800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00000001:1.0:1788572332.409052:0:58077:0:(import.c:672:import_select_connection()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788572332.409053:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 02000000:00000001:1.0:1788572332.409058:0:58077:0:(sec.c:1579:sptlrpc_import_sec_adapt()) Process entered 00000100:00000001:0.0:1788572332.409059:0:2384:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS new deadline estimation for cpu 2 is 4298927950 02000000:00000001:1.0:1788572332.409067:0:58077:0:(sec.c:1619:sptlrpc_import_sec_adapt()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788572332.409069:0:2384:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4298922830 00000100:00000001:0.0:1788572332.409073:0:2384:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS new deadline estimation for cpu 3 is 4298927950 00000100:00000001:0.0:1788572332.409077:0:2384:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4298922830 00000100:00000001:0.0:1788572332.409079:0:2384:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788572332.409083:0:2384:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572332.409086:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572332.409088:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572332.409097:0:2384:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788572332.409099:0:2384:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788572332.409101:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788572332.409103:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788572332.409104:0:2384:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788572332.409108:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada456bc000 x1875449616092416/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572332.409116:0:2384:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788572332.409117:0:2384:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572332.409119:0:2384:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788572332.409121:0:2384:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788572332.409123:0:2384:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788572332.409126:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada456bc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616092416:192.168.202.113@tcp:400:kworker.0 00000100:00000001:0.0:1788572332.409129:0:2384:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788572332.409131:0:2384:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788572332.409133:0:2384:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572332.409134:0:2384:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788572332.409139:0:2384:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada456bb400. 02000000:00000001:1.0:1788572332.409139:0:58077:0:(sec.c:1647:sptlrpc_import_sec_adapt()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572332.409144:0:2384:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572332.409145:0:58077:0:(obd_class.h:888:obd_reconnect()) Process entered 00000400:00000200:0.0:1788572332.409150:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000008:00100000:1.0:1788572332.409152:0:58077:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683394 ocd_grant: 8388608, lost: 0. 00000400:00000010:0.0:1788572332.409154:0:2384:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada60acaf20. 00000008:00000001:1.0:1788572332.409157:0:58077:0:(osc_request.c:3833:osc_reconnect()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788572332.409160:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada5176aaa0. 00000100:00000001:1.0:1788572332.409160:0:58077:0:(obd_class.h:902:obd_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1788572332.409165:0:2384:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616092416, portal 10 00000100:00000010:1.0:1788572332.409165:0:58077:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada456bdc00. 00000020:00000040:1.0:1788572332.409170:0:58077:0:(genops.c:1210:class_import_get()) import ffff9ada48407800 refcount=3 obd=lustre-OST0000-osc-ffff9ada58b0a000 00000100:00000001:0.0:1788572332.409171:0:2384:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788572332.409173:0:2384:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860991078400 : -111212718473216 : ffff9ada456bc000) 02000000:00000001:1.0:1788572332.409176:0:58077:0:(sec.c:2038:sptlrpc_sepol_get()) Process entered 02000000:00000001:1.0:1788572332.409179:0:58077:0:(sec.c:2048:sptlrpc_sepol_get()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572332.409183:0:2384:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada456bc000 x1875449616092416/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572348 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572332.409184:0:58077:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788572332.409187:0:58077:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788572332.409190:0:58077:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572332.409196:0:2384:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000010:1.0:1788572332.409198:0:58077:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 1024 at ffff9ada454b9400. 00000100:00000040:0.0:1788572332.409200:0:2384:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000100:00000001:1.0:1788572332.409204:0:58077:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1788572332.409205:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada5176ae58. 00000100:00000200:0.0:1788572332.409210:0:2384:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875449616092416, offset 0 00000100:00100000:1.0:1788572332.409215:0:58077:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada456bdc00 x1875449616092544/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000400:00000200:0.0:1788572332.409219:0:2384:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:0.0:1788572332.409227:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:1.0:1788572332.409230:0:58077:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788572332.409233:0:58077:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788572332.409234:0:2384:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:0.0:1788572332.409239:0:2384:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000040:1.0:1788572332.409240:0:58077:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000b2e0c703] to pc [ptlrpcd_rcv+-1] req@ffff9ada456bdc00 x1875449616092544/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788572332.409244:0:2384:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:0.0:1788572332.409248:0:2384:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788572332.409253:0:2384:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:0.0:1788572332.409259:0:2384:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22291:22291:255:4294967295] 192.168.202.113@tcp LPNI seq info [22287:22287:7:4294967295] 00000400:00000200:0.0:1788572332.409264:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:0.0:1788572332.409270:0:2384:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:0.0:1788572332.409274:0:2384:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:0.0:1788572332.409278:0:2384:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada5074a800. 00000800:00000200:0.0:1788572332.409283:0:2384:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:0.0:1788572332.409288:0:2384:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:0.0:1788572332.409291:0:2384:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada5074a800 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788572332.409294:0:2384:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572332.409295:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788572332.409298:0:2384:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: using a timeout of 16603 usecs (17 jiffies) 00000100:00000001:2.0:1788572332.409299:0:2382:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788572332.409302:0:2382:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000400:00000200:0.0:1788572332.409306:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:0.0:1788572332.409310:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298917395 00000100:00000001:0.0:1788572332.409313:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298927950 00000100:00000001:0.0:1788572332.409315:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298917395 00000100:00000001:0.0:1788572332.409316:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298927950 00000100:00000040:2.0:1788572332.409317:0:2382:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada456bdc00 x1875449616092544/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572332.409319:0:2384:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788572332.409324:0:2384:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572332.409330:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572332.409332:0:2382:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788572332.409334:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572332.409336:0:2382:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572332.409339:0:2384:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:2.0:1788572332.409340:0:2382:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:0.0:1788572332.409342:0:2384:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 02000000:00000001:2.0:1788572332.409344:0:2382:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788572332.409347:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:2.0:1788572332.409348:0:2382:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572332.409350:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00100000:2.0:1788572332.409353:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada456bdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616092544:192.168.202.113@tcp:8:kworker.0 00000100:00000001:2.0:1788572332.409356:0:2382:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788572332.409357:0:2382:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788572332.409359:0:2382:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572332.409362:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572332.409363:0:2382:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00100000:0.0:1788572332.409368:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00000001:0.0:1788572332.409370:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 02000000:00000010:2.0:1788572332.409370:0:2382:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at ffff9ada60568c00. 00000100:00000001:0.0:1788572332.409374:0:2384:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 02000000:00000001:2.0:1788572332.409375:0:2382:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572332.409377:0:2384:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572332.409381:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1788572332.409382:0:2382:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:0.0:1788572332.409383:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788572332.409386:0:2384:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000400:00000010:2.0:1788572332.409388:0:2382:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada43d96a50. 00000100:00000040:0.0:1788572332.409393:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada456bf800 x1875449616092288/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000010:2.0:1788572332.409394:0:2382:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada43c8faa0. 00000100:00000200:2.0:1788572332.409399:0:2382:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1875449616092544, portal 4 00000100:00000001:2.0:1788572332.409403:0:2382:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788572332.409407:0:2384:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788572332.409407:0:2382:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860991085568 : -111212718466048 : ffff9ada456bdc00) 00000100:00000001:0.0:1788572332.409410:0:2384:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572332.409414:0:2384:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788572332.409416:0:2384:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000040:2.0:1788572332.409418:0:2382:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada456bdc00 x1875449616092544/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572357 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1788572332.409419:0:2384:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788572332.409426:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada456bf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616092288:192.168.202.113@tcp:400:kworker.0 00000100:00000001:0.0:1788572332.409433:0:2384:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:2.0:1788572332.409433:0:2382:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:0.0:1788572332.409438:0:2384:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:2.0:1788572332.409440:0:2382:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 02000000:00000001:0.0:1788572332.409441:0:2384:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572332.409445:0:2384:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000010:2.0:1788572332.409446:0:2382:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada43c8fdd0. 02000000:00000010:0.0:1788572332.409449:0:2384:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada456bb800. 00000100:00000200:2.0:1788572332.409451:0:2382:0:(niobuf.c:94:ptl_send_buf()) Sending 520 bytes to portal 28, xid 1875449616092544, offset 0 02000000:00000001:0.0:1788572332.409454:0:2384:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788572332.409461:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572332.409462:0:2382:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000010:0.0:1788572332.409466:0:2384:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada60aca108. 00000400:00000200:2.0:1788572332.409467:0:2382:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572332.409472:0:2382:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000010:0.0:1788572332.409473:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada5176a110. 00000400:00000200:2.0:1788572332.409476:0:2382:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000200:0.0:1788572332.409479:0:2384:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616092288, portal 10 00000400:00000200:2.0:1788572332.409479:0:2382:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000100:00000001:0.0:1788572332.409484:0:2384:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000400:00000200:2.0:1788572332.409484:0:2382:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1788572332.409487:0:2384:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860991092736 : -111212718458880 : ffff9ada456bf800) 00000400:00000200:2.0:1788572332.409488:0:2382:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:2.0:1788572332.409492:0:2382:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22292:22292:254:4294967295] 192.168.202.113@tcp LPNI seq info [22288:22288:6:4294967295] 00000400:00000200:2.0:1788572332.409497:0:2382:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000040:0.0:1788572332.409498:0:2384:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada456bf800 x1875449616092288/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572348 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1788572332.409507:0:2382:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:2.0:1788572332.409518:0:2382:0:(socklnd_cb.c:931:ksocknal_send()) sending 520 bytes in 1 frags to 12345-192.168.202.113@tcp 00000100:00000001:0.0:1788572332.409520:0:2384:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000010:2.0:1788572332.409523:0:2382:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada509ed000. 00000100:00000040:0.0:1788572332.409527:0:2384:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000800:00000200:2.0:1788572332.409530:0:2382:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000400:00000010:0.0:1788572332.409532:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada5176a908. 00000800:00000200:2.0:1788572332.409537:0:2382:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000100:00000200:0.0:1788572332.409538:0:2384:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875449616092288, offset 0 00000800:00000200:2.0:1788572332.409540:0:2382:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada509ed000 type 1, nob 616 niov 1 nkiov 1 00000100:00000001:2.0:1788572332.409544:0:2382:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572332.409546:0:2382:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: using a timeout of 18878 usecs (19 jiffies) 00000400:00000200:0.0:1788572332.409550:0:2384:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:2.0:1788572332.409551:0:2382:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:2.0:1788572332.409554:0:2382:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298917397 00000100:00000001:2.0:1788572332.409556:0:2382:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298927950 00000100:00000001:2.0:1788572332.409557:0:2382:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298917397 00000100:00000001:2.0:1788572332.409558:0:2382:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298927950 00000100:00000001:2.0:1788572332.409559:0:2382:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788572332.409561:0:2382:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572332.409562:0:2382:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572332.409572:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:3.0:1788572332.409692:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788572332.409697:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada5074a400. 00000400:00000200:3.0:1788572332.409706:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572332.409712:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:0.0:1788572332.409872:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:0.0:1788572332.409878:0:2384:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000100:00000001:1.0:1788572332.409880:0:58077:0:(import.c:877:ptlrpc_connect_import_locked()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788572332.409881:0:2384:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788572332.409886:0:2384:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000100:00000040:1.0:1788572332.409887:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000200:0.0:1788572332.409890:0:2384:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000100:00000040:1.0:1788572332.409895:0:58077:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000200:0.0:1788572332.409896:0:2384:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:0.0:1788572332.409902:0:2384:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22293:22293:253:4294967295] 192.168.202.113@tcp LPNI seq info [22289:22289:5:4294967295] 00000100:00000040:1.0:1788572332.409905:0:58077:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (4255) 00000400:00000200:0.0:1788572332.409908:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:0.0:1788572332.409917:0:2384:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:0.0:1788572332.409923:0:2384:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:0.0:1788572332.409927:0:2384:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada5074aa00. 00000800:00000200:0.0:1788572332.409931:0:2384:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:0.0:1788572332.409935:0:2384:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:0.0:1788572332.409938:0:2384:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada5074aa00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788572332.409941:0:2384:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572332.409942:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:0.0:1788572332.409945:0:2384:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: using a timeout of 41268 usecs (42 jiffies) 00000100:00000001:1.0:1788572332.409946:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788572332.409951:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:0.0:1788572332.409954:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298917421 00000100:00000001:0.0:1788572332.409957:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298927950 00000100:00000001:0.0:1788572332.409959:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298917421 00000100:00000001:0.0:1788572332.409961:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298927950 00000100:00000001:0.0:1788572332.409964:0:2384:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788572332.409966:0:2384:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572332.409974:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572332.409977:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572332.410007:0:2382:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788572332.410010:0:2382:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=5 : 5 : 5) 00000100:00000001:2.0:1788572332.410012:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788572332.410014:0:2382:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788572332.410015:0:2382:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572332.410017:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788572332.415251:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada5176a088 00000400:00000010:3.0:1788572332.415269:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada5176a088. 00000100:00000001:3.0:1788572332.415283:0:2377:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:3.0:1788572332.415298:0:2377:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada456bce00 x1875449616092160/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572348 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572332.415312:0:2377:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572332.415316:0:2377:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada456bce00 x1875449616092160/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572348 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572332.415329:0:2377:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572332.415331:0:2377:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000200:3.0:1788572332.416871:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788572332.416886:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada5074a800. 00000400:00000200:3.0:1788572332.416898:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572332.416908:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:3.0:1788572332.416916:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada5176ae58 00000400:00000010:3.0:1788572332.416919:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada5176ae58. 00000100:00000001:3.0:1788572332.416925:0:2377:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:3.0:1788572332.416935:0:2377:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada456bc000 x1875449616092416/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572348 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572332.416949:0:2377:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572332.416953:0:2377:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada456bc000 x1875449616092416/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572348 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572332.416959:0:2377:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572332.416961:0:2377:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000200:3.0:1788572332.416990:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788572332.416993:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada509ed000. 00000400:00000200:3.0:1788572332.416996:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572332.417000:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:3.0:1788572332.417003:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada43c8fdd0 00000400:00000010:3.0:1788572332.417005:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada43c8fdd0. 00000100:00000001:3.0:1788572332.417008:0:2377:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:3.0:1788572332.417012:0:2377:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada456bdc00 x1875449616092544/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572357 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572332.417020:0:2377:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572332.417023:0:2377:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada456bdc00 x1875449616092544/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572357 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572332.417029:0:2377:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572332.417031:0:2377:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000200:3.0:1788572332.417144:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788572332.417148:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada5074aa00. 00000400:00000200:3.0:1788572332.417152:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572332.417156:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:3.0:1788572332.417159:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada5176a908 00000400:00000010:3.0:1788572332.417161:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada5176a908. 00000100:00000001:3.0:1788572332.417164:0:2377:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:3.0:1788572332.417168:0:2377:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada456bf800 x1875449616092288/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572348 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572332.417177:0:2377:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572332.417179:0:2377:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada456bf800 x1875449616092288/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572348 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572332.417185:0:2377:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572332.417186:0:2377:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000001:3.0:1788572332.429925:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572332.429955:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572332.429961:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572332.429964:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572332.429975:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572332.429996:0:2378:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572332.430008:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572332.430013:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572332.430020:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572332.430025:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 25 MB=0x6a9b624294c00 00000400:00000200:3.0:1788572332.430032:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572332.430040:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.202.113@tcp of length 224/224 into md 0x35785 [1] + 224 00000800:00000001:3.0:1788572332.430050:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572332.430059:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572332.430064:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572332.430070:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572332.430082:0:2378:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572332.430094:0:2378:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada456bce00 x1875449616092160/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572348 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572332.430107:0:2378:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada456bce00 x1875449616092160/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572348 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572332.430136:0:2378:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572332.430149:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572332.430153:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572332.430552:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788572332.430568:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788572332.430579:0:2384:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788572332.430581:0:2384:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788572332.430585:0:2384:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada60acacb8. 00000400:00000200:0.0:1788572332.430592:0:2384:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada5176a770 00000400:00000010:0.0:1788572332.430595:0:2384:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada5176a770. 00000100:00000001:0.0:1788572332.430598:0:2384:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:0.0:1788572332.430605:0:2384:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada456bce00 x1875449616092160/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572348 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788572332.430614:0:2384:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada456bce00 x1875449616092160/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572348 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572332.430620:0:2384:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788572332.430622:0:2384:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572332.430625:0:2384:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:0.0:1788572332.430629:0:2384:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788572332.430633:0:2384:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788572332.430635:0:2384:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572332.430638:0:2384:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788572332.430648:0:2384:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada58a62c60 time=94 v=5 (5 5 5 5) 00000100:00001000:0.0:1788572332.430653:0:2384:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada58a62c18 time=124 v=5 (1 0 0 0) 00000100:00000001:0.0:1788572332.430655:0:2384:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788572332.430658:0:2384:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788572332.430662:0:2384:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788572332.430664:0:2384:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572332.430667:0:2384:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572332.430671:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada456bce00 x1875449616092160/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572348 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572332.430678:0:2384:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788572332.430682:0:2384:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788572332.430684:0:2384:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788572332.430688:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada456bce00 x1875449616092160/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572348 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572332.430695:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada456bce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449616092160:192.168.202.113@tcp:400:kworker.0 00000100:00000001:0.0:1788572332.430701:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572332.430704:0:2384:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788572332.430708:0:2384:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada456bce00 x1875449616092160/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572348 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572332.430716:0:2384:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788572332.430719:0:2384:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788572332.430724:0:2384:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada456bba00. 02000000:00000001:0.0:1788572332.430729:0:2384:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788572332.430740:0:2384:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:0.0:1788572332.430742:0:2384:0:(genops.c:1223:class_import_put()) import ffff9ada58a62800 refcount=3 obd=MGC192.168.202.113@tcp 00000020:00000001:0.0:1788572332.430744:0:2384:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:0.0:1788572332.430748:0:2384:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada48e6fa00. 02000000:00000001:0.0:1788572332.430752:0:2384:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788572332.430755:0:2384:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788572332.430757:0:2384:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada456bce00. 00000100:00000001:0.0:1788572332.430762:0:2384:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788572332.430763:0:2384:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572332.430767:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572332.430770:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788572332.430771:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788572332.430773:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572332.430775:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788572332.432456:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572332.432487:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572332.432492:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572332.432495:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572332.432509:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572332.432519:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572332.432532:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572332.432537:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572332.432545:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572332.432551:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 4 MB=0x6a9b624294d80 00000400:00000200:3.0:1788572332.432567:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572332.432574:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.202.113@tcp of length 224/224 into md 0x35795 [1] + 0 00000800:00000001:3.0:1788572332.432584:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572332.432652:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572332.432657:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572332.432664:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572332.432674:0:2377:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572332.432684:0:2377:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada456bdc00 x1875449616092544/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572357 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572332.432696:0:2377:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=0 replen=544 req@ffff9ada456bdc00 x1875449616092544/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572357 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572332.432724:0:2377:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572332.432743:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572332.432747:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572332.435209:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788572332.435221:0:2382:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788572332.435234:0:2382:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788572332.435237:0:2382:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788572332.435242:0:2382:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada43d96a50. 00000400:00000200:2.0:1788572332.435253:0:2382:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada43c8faa0 00000400:00000010:2.0:1788572332.435256:0:2382:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada43c8faa0. 00000100:00000001:2.0:1788572332.435261:0:2382:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:2.0:1788572332.435276:0:2382:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada456bdc00 x1875449616092544/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572357 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788572332.435288:0:2382:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada456bdc00 x1875449616092544/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572357 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572332.435301:0:2382:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788572332.435303:0:2382:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572332.435306:0:2382:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:2.0:1788572332.435311:0:2382:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788572332.435325:0:2382:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788572332.435328:0:2382:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572332.435331:0:2382:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788572332.435342:0:2382:0:(import.c:2195:obd_at_measure()) add 0 to ffff9ada48407c60 time=177 v=5 (5 5 5 5) 00000100:00001000:2.0:1788572332.435345:0:2382:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada48407c18 time=54 v=20 (20 1 1 1) 00000100:00000001:2.0:1788572332.435349:0:2382:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788572332.435351:0:2382:0:(client.c:1506:ptlrpc_check_status()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000001:2.0:1788572332.435354:0:2382:0:(client.c:1712:after_reply()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000040:2.0:1788572332.435359:0:2382:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada456bdc00 x1875449616092544/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/224 e 0 to 0 dl 1788572357 ref 1 fl Rpc:RNQU/200/0 rc -19/-19 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572332.435367:0:2382:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=18446744073709551597 : -19 : 0xffffffffffffffed) 00000100:00000001:2.0:1788572332.435371:0:2382:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788572332.435374:0:2382:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572332.435378:0:2382:0:(import.c:1101:ptlrpc_connect_interpret()) Process entered 00000100:00000001:2.0:1788572332.435381:0:2382:0:(import.c:1136:ptlrpc_connect_interpret()) Process leaving via out (rc=18446744073709551597 : -19 : 0xffffffffffffffed) 00000100:00080000:2.0:1788572332.435385:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada48407800 lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788572332.435391:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.113@tcp failed (-19) 00000100:00000001:2.0:1788572332.435393:0:2382:0:(import.c:1561:ptlrpc_connect_interpret()) Process leaving (rc=18446744073709551597 : -19 : ffffffffffffffed) 00000100:00000040:2.0:1788572332.435397:0:2382:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada456bdc00 x1875449616092544/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/224 e 0 to 0 dl 1788572357 ref 1 fl Interpret:RNQU/200/0 rc -19/-19 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788572332.435413:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada456bdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616092544:192.168.202.113@tcp:8:kworker.0 00000100:00000001:2.0:1788572332.435419:0:2382:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572332.435422:0:2382:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572332.435425:0:2382:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada456bdc00 x1875449616092544/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/224 e 0 to 0 dl 1788572357 ref 1 fl Complete:RNQU/200/0 rc -19/-19 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572332.435437:0:2382:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572332.435440:0:2382:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572332.435444:0:2382:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada60568c00. 02000000:00000001:2.0:1788572332.435449:0:2382:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572332.435452:0:2382:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572332.435454:0:2382:0:(genops.c:1223:class_import_put()) import ffff9ada48407800 refcount=2 obd=lustre-OST0000-osc-ffff9ada58b0a000 00000020:00000001:2.0:1788572332.435457:0:2382:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572332.435461:0:2382:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 1024 at ffff9ada454b9400. 02000000:00000001:2.0:1788572332.435469:0:2382:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572332.435471:0:2382:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572332.435473:0:2382:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada456bdc00. 00000100:00000001:2.0:1788572332.435477:0:2382:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572332.435478:0:2382:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572332.435481:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572332.435490:0:2382:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788572332.435492:0:2382:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572332.435494:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788572332.435496:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788572332.435571:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572332.435589:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572332.435593:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572332.435595:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572332.435605:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572332.435610:0:2378:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572332.435621:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572332.435626:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572332.435632:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572332.435637:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 10 MB=0x6a9b624294d00 00000400:00000200:3.0:1788572332.435642:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572332.435648:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.113@tcp of length 224/224 into md 0x3578d [1] + 224 00000800:00000001:3.0:1788572332.435658:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572332.435663:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572332.435667:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572332.435672:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572332.435681:0:2378:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572332.435687:0:2378:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada456bc000 x1875449616092416/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572348 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572332.435697:0:2378:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada456bc000 x1875449616092416/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572348 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572332.435718:0:2378:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572332.435729:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572332.435742:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572332.435744:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572332.435747:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572332.435750:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572332.435754:0:2378:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572332.435758:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572332.435760:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572332.435763:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572332.435766:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 10 MB=0x6a9b624294c80 00000400:00000200:3.0:1788572332.435769:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572332.435773:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.113@tcp of length 224/224 into md 0x357a5 [1] + 224 00000800:00000001:3.0:1788572332.435776:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572332.435786:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572332.435788:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572332.435792:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572332.435795:0:2378:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572332.435813:0:2378:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada456bf800 x1875449616092288/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572348 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572332.435823:0:2378:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada456bf800 x1875449616092288/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572348 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572332.435831:0:2378:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572332.435836:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572332.435840:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572332.436028:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788572332.436032:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788572332.436052:0:2384:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788572332.436054:0:2384:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788572332.436058:0:2384:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada60aca108. 00000400:00000200:0.0:1788572332.436062:0:2384:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada5176a110 00000400:00000010:0.0:1788572332.436064:0:2384:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada5176a110. 00000100:00000001:0.0:1788572332.436067:0:2384:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:0.0:1788572332.436073:0:2384:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada456bf800 x1875449616092288/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572348 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788572332.436083:0:2384:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada456bf800 x1875449616092288/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572348 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572332.436089:0:2384:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788572332.436091:0:2384:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572332.436093:0:2384:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:0.0:1788572332.436095:0:2384:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788572332.436097:0:2384:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788572332.436099:0:2384:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572332.436101:0:2384:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788572332.436108:0:2384:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada51140460 time=27 v=5 (5 5 5 5) 00000100:00001000:0.0:1788572332.436111:0:2384:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada51140418 time=123 v=5 (2 0 0 0) 00000100:00000001:0.0:1788572332.436114:0:2384:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788572332.436116:0:2384:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788572332.436119:0:2384:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788572332.436126:0:2384:0:(ldlm_request.c:1703:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572332.436130:0:2384:0:(client.c:3110:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788572332.436133:0:2384:0:(client.c:3122:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: skip recheck: last_committed 219043334508 00000100:00000001:0.0:1788572332.436135:0:2384:0:(client.c:3123:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788572332.436137:0:2384:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572332.436141:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada456bf800 x1875449616092288/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572348 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572332.436148:0:2384:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788572332.436151:0:2384:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788572332.436153:0:2384:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788572332.436156:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada456bf800 x1875449616092288/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572348 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572332.436164:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada456bf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616092288:192.168.202.113@tcp:400:kworker.0 00000400:00000001:0.0:1788572332.436170:0:2384:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788572332.436172:0:2384:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788572332.436173:0:2384:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada60acaf20. 00000400:00000200:0.0:1788572332.436176:0:2384:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada5176aaa0 00000400:00000010:0.0:1788572332.436178:0:2384:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada5176aaa0. 00000100:00000001:0.0:1788572332.436180:0:2384:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:0.0:1788572332.436183:0:2384:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada456bc000 x1875449616092416/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572348 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788572332.436191:0:2384:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada456bc000 x1875449616092416/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572348 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572332.436196:0:2384:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788572332.436198:0:2384:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572332.436200:0:2384:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:0.0:1788572332.436201:0:2384:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788572332.436202:0:2384:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788572332.436204:0:2384:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572332.436206:0:2384:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788572332.436209:0:2384:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada58b08460 time=94 v=5 (5 5 5 5) 00000100:00001000:0.0:1788572332.436212:0:2384:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada58b08418 time=94 v=5 (1 1 1 1) 00000100:00000001:0.0:1788572332.436215:0:2384:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788572332.436216:0:2384:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788572332.436218:0:2384:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788572332.436221:0:2384:0:(ldlm_request.c:1703:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572332.436223:0:2384:0:(client.c:3110:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788572332.436224:0:2384:0:(client.c:3122:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff9ada58b0a000: skip recheck: last_committed 0 00000100:00000001:0.0:1788572332.436226:0:2384:0:(client.c:3123:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788572332.436227:0:2384:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572332.436231:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada456bc000 x1875449616092416/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572348 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572332.436238:0:2384:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788572332.436240:0:2384:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788572332.436241:0:2384:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788572332.436244:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada456bc000 x1875449616092416/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572348 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572332.436251:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada456bc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616092416:192.168.202.113@tcp:400:kworker.0 00000100:00000001:0.0:1788572332.436255:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572332.436257:0:2384:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788572332.436260:0:2384:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada456bf800 x1875449616092288/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572348 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572332.436265:0:2384:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788572332.436267:0:2384:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788572332.436269:0:2384:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada456bb800. 02000000:00000001:0.0:1788572332.436273:0:2384:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788572332.436275:0:2384:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:0.0:1788572332.436277:0:2384:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=3 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:0.0:1788572332.436279:0:2384:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:0.0:1788572332.436281:0:2384:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada48e6f400. 02000000:00000001:0.0:1788572332.436284:0:2384:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788572332.436285:0:2384:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788572332.436287:0:2384:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada456bf800. 00000100:00000001:0.0:1788572332.436290:0:2384:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788572332.436291:0:2384:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572332.436292:0:2384:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788572332.436295:0:2384:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada456bc000 x1875449616092416/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572348 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572332.436300:0:2384:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788572332.436302:0:2384:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788572332.436303:0:2384:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada456bb400. 02000000:00000001:0.0:1788572332.436305:0:2384:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788572332.436306:0:2384:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:0.0:1788572332.436307:0:2384:0:(genops.c:1223:class_import_put()) import ffff9ada58b08000 refcount=3 obd=lustre-MDT0001-mdc-ffff9ada58b0a000 00000020:00000001:0.0:1788572332.436308:0:2384:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:0.0:1788572332.436310:0:2384:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada48e6f100. 02000000:00000001:0.0:1788572332.436311:0:2384:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788572332.436313:0:2384:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788572332.436314:0:2384:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada456bc000. 00000100:00000001:0.0:1788572332.436316:0:2384:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788572332.436317:0:2384:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572332.436319:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572332.436325:0:2384:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788572332.436326:0:2384:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572332.436328:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788572332.436330:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572332.919103:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572332.919114:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572332.919122:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572332.919123:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572332.919126:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572332.919128:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572332.919132:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572332.919136:0:365:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:2.0:1788572332.919139:0:365:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:2.0:1788572332.919148:0:365:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:2.0:1788572332.919150:0:365:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:2.0:1788572332.919152:0:365:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572332.919156:0:365:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:2.0:1788572332.919158:0:365:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572332.919161:0:365:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572332.919163:0:365:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572332.919705:0:15205:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572332.919709:0:15205:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572332.919714:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000800:00000001:2.0:1788572333.889644:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572333.889669:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788572333.889676:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572333.889680:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572333.889692:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572333.889702:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:2.0:1788572333.889714:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572333.889720:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572333.889726:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572333.889732:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 328 into portal 15 MB=0x6a9b625885980 00000400:00000200:2.0:1788572333.889743:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index f from 12345-192.168.202.113@tcp of length 328/328 into md 0x2df25 [2] + 0 00000800:00000001:2.0:1788572333.889752:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572333.889761:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572333.889766:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572333.889782:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:2.0:1788572333.889793:0:2377:0:(events.c:290:request_in_callback()) Process entered 00000100:00000200:2.0:1788572333.889796:0:2377:0:(events.c:300:request_in_callback()) event type 2, status 0, service ldlm_cbd 00000100:00000010:2.0:1788572333.889804:0:2377:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada60624000. 00000100:00000040:2.0:1788572333.889810:0:2377:0:(events.c:341:request_in_callback()) incoming req@ffff9ada60624000 x1875449639098752 msgsize 328 00000100:00100000:2.0:1788572333.889823:0:2377:0:(events.c:344:request_in_callback()) peer: 12345-192.168.202.113@tcp (source: 12345-192.168.202.113@tcp) 00000100:00000001:2.0:1788572333.889842:0:2377:0:(events.c:377:request_in_callback()) Process leaving 00000800:00000001:2.0:1788572333.889881:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572333.889885:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788572333.890418:0:33410:0:(service.c:2282:ptlrpc_server_handle_req_in()) Process entered 00000100:00100000:3.0:1788572333.890423:0:33410:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875449639098752 02000000:00000001:3.0:1788572333.890428:0:33410:0:(sec.c:2469:sptlrpc_svc_unwrap_request()) Process entered 00000100:00000001:3.0:1788572333.890431:0:33410:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788572333.890434:0:33410:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788572333.890439:0:33410:0:(sec.c:2518:sptlrpc_svc_unwrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788572333.890446:0:33410:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875449639098752 00000020:00000001:3.0:1788572333.890449:0:33410:0:(genops.c:792:class_conn2export()) Process entered 00000020:00000040:3.0:1788572333.890451:0:33410:0:(genops.c:804:class_conn2export()) looking for export cookie 0xef30c340bfb39c1a 00000020:00000001:3.0:1788572333.890455:0:33410:0:(lustre_handles.c:130:class_handle2object()) Process entered 00000020:00000040:3.0:1788572333.890459:0:33410:0:(lustre_handles.c:148:class_handle2object()) GET export ffff9ada51140800 refcount=3 00000020:00000001:3.0:1788572333.890462:0:33410:0:(lustre_handles.c:155:class_handle2object()) Process leaving (rc=18446632861186656256 : -111212522895360 : ffff9ada51140800) 00000020:00000001:3.0:1788572333.890464:0:33410:0:(genops.c:806:class_conn2export()) Process leaving (rc=18446632861186656256 : -111212522895360 : ffff9ada51140800) 00000100:00000001:3.0:1788572333.890470:0:33410:0:(service.c:1268:ptlrpc_update_export_timer()) Process entered 00000020:00000010:3.0:1788572333.890478:0:33410:0:(genops.c:1055:obd_export_timed_init()) kmalloced '(*data)': 48 at ffff9ada60789bc0. 00000100:00000001:3.0:1788572333.890481:0:33410:0:(service.c:1290:ptlrpc_update_export_timer()) Process leaving via err (rc=0 : 0 : 0x0) 00000020:00000010:3.0:1788572333.890484:0:33410:0:(genops.c:1063:obd_export_timed_fini()) kfreed '*data': 48 at ffff9ada60789bc0. 00000020:00000010:3.0:1788572333.890491:0:33410:0:(lu_object.c:1758:keys_init()) kmalloced '(ctx->lc_value)': 320 at ffff9ada50a3b400. 00000020:00000010:3.0:1788572333.890508:0:33410:0:(lu_ucred.c:27:lu_ucred_key_init()) kmalloced '(value)': 128 at ffff9ada60aaa600. 00000020:00000010:3.0:1788572333.890515:0:33410:0:(tgt_main.c:758:tgt_ses_key_init()) slab-alloced 'session': 200 at ffff9ada4890fbb8. 00000100:00000040:3.0:1788572333.890526:0:33410:0:(service.c:1432:ptlrpc_at_set_timer()) armed ldlm_cbd at +6s 00000100:00000001:3.0:1788572333.890530:0:33410:0:(service.c:2003:ptlrpc_server_request_add()) Process entered 00000100:00000001:3.0:1788572333.890533:0:33410:0:(service.c:1919:ptlrpc_server_hpreq_init()) Process entered 00000100:00000001:3.0:1788572333.890535:0:33410:0:(service.c:1949:ptlrpc_server_hpreq_init()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572333.890545:0:33410:0:(service.c:2101:ptlrpc_server_request_add()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572333.890561:0:33410:0:(service.c:2445:ptlrpc_server_handle_req_in()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788572333.890574:0:33410:0:(service.c:2479:ptlrpc_server_handle_request()) Process entered 00000100:00000001:3.0:1788572333.890576:0:33410:0:(service.c:2223:ptlrpc_server_request_get()) Process entered 00000100:00100000:3.0:1788572333.890582:0:33410:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.113@tcp, seq: 238 00000100:00000040:3.0:1788572333.890586:0:33410:0:(service.c:2255:ptlrpc_server_request_get()) RPC GETting export ffff9ada51140800 : new rpc_count 1 00000100:00000001:3.0:1788572333.890589:0:33410:0:(service.c:2257:ptlrpc_server_request_get()) Process leaving (rc=18446632861443440640 : -111212266110976 : ffff9ada60624000) 00000100:00000040:3.0:1788572333.890598:0:33410:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Interpret req@ffff9ada60624000 x1875449639098752/t0(0) o104->90fa6266-5a39-4f6d-b3cf-5043b0cc0e22@192.168.202.113@tcp:749/0 lens 328/0 e 0 to 0 dl 1788572344 ref 1 fl New:/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788572333.890608:0:33410:0:(service.c:1268:ptlrpc_update_export_timer()) Process entered 00000020:00000010:3.0:1788572333.890612:0:33410:0:(genops.c:1055:obd_export_timed_init()) kmalloced '(*data)': 48 at ffff9ada60789bc0. 00000100:00000001:3.0:1788572333.890614:0:33410:0:(service.c:1290:ptlrpc_update_export_timer()) Process leaving via err (rc=0 : 0 : 0x0) 00000020:00000010:3.0:1788572333.890616:0:33410:0:(genops.c:1063:obd_export_timed_fini()) kfreed '*data': 48 at ffff9ada60789bc0. 00000100:00100000:3.0:1788572333.890620:0:33410:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9ada60624000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_002:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22+3:111988:x1875449639098752:12345-192.168.202.113@tcp:104: 00000100:00000200:3.0:1788572333.890624:0:33410:0:(service.c:2563:ptlrpc_server_handle_request()) got req 1875449639098752 00010000:00000001:3.0:1788572333.890627:0:33410:0:(ldlm_lockd.c:2360:ldlm_callback_handler()) Process entered 00010000:00000001:3.0:1788572333.890635:0:33410:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:3.0:1788572333.890637:0:33410:0:(lustre_handles.c:130:class_handle2object()) Process entered 00000020:00000040:3.0:1788572333.890639:0:33410:0:(lustre_handles.c:148:class_handle2object()) GET ldlm ffff9ada4542aa00 refcount=2 00000020:00000001:3.0:1788572333.890641:0:33410:0:(lustre_handles.c:155:class_handle2object()) Process leaving (rc=18446632860988385792 : -111212721165824 : ffff9ada4542aa00) 00010000:00000001:3.0:1788572333.890645:0:33410:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446632860988385792 : -111212721165824 : ffff9ada4542aa00) 00010000:00000001:3.0:1788572333.890649:0:33410:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:3.0:1788572333.890653:0:33410:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788572333.890659:0:33410:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.202.113@tcp lock: ffff9ada4542aa00/0xef30c340bfb45a40 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1400000000000 nid: local remote: 0xdb7edb52b831ca7a expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00000001:3.0:1788572333.890671:0:33410:0:(pack_generic.c:297:lustre_pack_reply_v2()) Process entered 02000000:00000001:3.0:1788572333.890674:0:33410:0:(sec.c:2540:sptlrpc_svc_alloc_rs()) Process entered 02000000:00000010:3.0:1788572333.890679:0:33410:0:(sec_null.c:305:null_alloc_rs()) kmalloced '(rs)': 520 at ffff9ada607bc000. 02000000:00000001:3.0:1788572333.890682:0:33410:0:(sec.c:2577:sptlrpc_svc_alloc_rs()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572333.890684:0:33410:0:(pack_generic.c:332:lustre_pack_reply_v2()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788572333.890690:0:33410:0:(ldlm_lib.c:3345:target_pack_pool_reply()) Process entered 00010000:00000001:3.0:1788572333.890693:0:33410:0:(ldlm_lib.c:3355:target_pack_pool_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788572333.890697:0:33410:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada43582de8 time=172 v=5 (1 0 1 1) 00000100:00000001:3.0:1788572333.890703:0:33410:0:(connection.c:157:ptlrpc_connection_addref()) Process entered 00000100:00000040:3.0:1788572333.890706:0:33410:0:(connection.c:162:ptlrpc_connection_addref()) conn=ffff9ada51db1200 refcount 16 to 192.168.202.113@tcp 00000100:00000001:3.0:1788572333.890708:0:33410:0:(connection.c:164:ptlrpc_connection_addref()) Process leaving (rc=18446632861199700480 : -111212509851136 : ffff9ada51db1200) 02000000:00000001:3.0:1788572333.890711:0:33410:0:(sec.c:2591:sptlrpc_svc_wrap_reply()) Process entered 02000000:00000001:3.0:1788572333.890714:0:33410:0:(sec.c:2602:sptlrpc_svc_wrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572333.890718:0:33410:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788572333.890721:0:33410:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:3.0:1788572333.890726:0:33410:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada51269f68. 00000100:00000200:3.0:1788572333.890732:0:33410:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 16, xid 1875449639098752, offset 224 00000400:00000200:3.0:1788572333.890740:0:33410:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:3.0:1788572333.890752:0:33410:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572333.890760:0:33410:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source Specified: 192.168.202.13@tcp to MR: 192.168.202.113@tcp local destination 00000400:00000200:3.0:1788572333.890767:0:33410:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22294:22294:256:4294967295] 192.168.202.113@tcp LPNI seq info [22290:22290:8:4294967295] 00000400:00000200:3.0:1788572333.890783:0:33410:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572333.890792:0:33410:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:192.168.202.13@tcp) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:3.0:1788572333.890797:0:33410:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:3.0:1788572333.890802:0:33410:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada50c31a00. 00000800:00000200:3.0:1788572333.890810:0:33410:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:3.0:1788572333.891035:0:33410:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:3.0:1788572333.891041:0:33410:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada50c31a00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788572333.891093:0:33410:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572333.891098:0:33410:0:(lustre_net.h:1977:ptlrpc_connection_put()) PUT conn=ffff9ada51db1200 refcount 15 to 192.168.202.113@tcp 00010000:00000001:3.0:1788572333.891101:0:33410:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000010:3.0:1788572333.891106:0:33410:0:(ldlm_lockd.c:2249:ldlm_bl_to_thread()) kmalloced '(blwi)': 192 at ffff9ada4352f6c0. 00010000:00000001:3.0:1788572333.891110:0:33410:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) Process entered 00010000:00010000:3.0:1788572333.891126:0:33410:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00010000:00000001:3.0:1788572333.891129:0:33410:0:(ldlm_lockd.c:2198:__ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788572333.891131:0:33410:0:(ldlm_lockd.c:2254:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788572333.891134:0:33410:0:(ldlm_lockd.c:2533:ldlm_callback_handler()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572333.891140:0:33410:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada60624000 x1875449639098752/t0(0) o104->90fa6266-5a39-4f6d-b3cf-5043b0cc0e22@192.168.202.113@tcp:749/0 lens 328/224 e 0 to 0 dl 1788572344 ref 1 fl Interpret:/0/0 rc 0/0 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788572333.891151:0:33410:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9ada60624000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_002:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22+3:111988:x1875449639098752:12345-192.168.202.113@tcp:104: Request processed in 541us (1340us total) trans 0 rc 0/0 00000100:00100000:3.0:1788572333.891161:0:33410:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.113@tcp, seq: 238 00000100:00000040:3.0:1788572333.891165:0:33410:0:(service.c:1113:ptlrpc_server_finish_active_request()) RPC PUTting export ffff9ada51140800 : new rpc_count 0 00000100:00000001:3.0:1788572333.891167:0:33410:0:(service.c:1955:ptlrpc_server_hpreq_fini()) Process entered 00000100:00000001:3.0:1788572333.891169:0:33410:0:(service.c:1966:ptlrpc_server_hpreq_fini()) Process leaving 00000020:00000010:3.0:1788572333.891175:0:33410:0:(lu_ucred.c:27:lu_ucred_key_fini()) kfreed 'info': 128 at ffff9ada60aaa600. 00000020:00000010:3.0:1788572333.891180:0:33410:0:(tgt_main.c:770:tgt_ses_key_fini()) slab-freed '(session)': 200 at ffff9ada4890fbb8. 00000020:00000010:3.0:1788572333.891184:0:33410:0:(lu_object.c:1686:keys_fini()) kfreed 'ctx->lc_value': 320 at ffff9ada50a3b400. 00000020:00000040:3.0:1788572333.891190:0:33410:0:(genops.c:879:class_export_put()) PUTting export ffff9ada51140800 : new refcount 2 00000100:00000001:3.0:1788572333.891193:0:33410:0:(service.c:2635:ptlrpc_server_handle_request()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:0.0:1788572333.891333:0:15204:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572333.891337:0:15204:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572333.891340:0:15204:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00000001:0.0:1788572333.891343:0:15204:0:(ldlm_lockd.c:2881:ldlm_bl_thread_blwi()) Process entered 00010000:00000001:0.0:1788572333.891346:0:15204:0:(ldlm_lockd.c:1905:ldlm_handle_bl_callback()) Process entered 00010000:00010000:0.0:1788572333.891353:0:15204:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.202.113@tcp lock: ffff9ada4542aa00/0xef30c340bfb45a40 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1400000000000 nid: local remote: 0xdb7edb52b831ca7a expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572333.891362:0:15204:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada4542aa00 already unused, calling callback (ffffffffc11b5c20) 10000000:00000001:0.0:1788572333.891365:0:15204:0:(mgc_request.c:917:mgc_blocking_ast()) Process entered 10000000:00010000:0.0:1788572333.891368:0:15204:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.202.113@tcp lock: ffff9ada4542aa00/0xef30c340bfb45a40 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1400400000000 nid: local remote: 0xdb7edb52b831ca7a expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:0.0:1788572333.891375:0:15204:0:(ldlm_request.c:1770:ldlm_cli_cancel()) Process entered 00010000:00000001:0.0:1788572333.891377:0:15204:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:0.0:1788572333.891378:0:15204:0:(lustre_handles.c:130:class_handle2object()) Process entered 00000020:00000040:0.0:1788572333.891380:0:15204:0:(lustre_handles.c:148:class_handle2object()) GET ldlm ffff9ada4542aa00 refcount=3 00000020:00000001:0.0:1788572333.891382:0:15204:0:(lustre_handles.c:155:class_handle2object()) Process leaving (rc=18446632860988385792 : -111212721165824 : ffff9ada4542aa00) 00010000:00000001:0.0:1788572333.891384:0:15204:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446632860988385792 : -111212721165824 : ffff9ada4542aa00) 00010000:00000001:0.0:1788572333.891388:0:15204:0:(ldlm_request.c:1386:ldlm_cli_cancel_local()) Process entered 00010000:00010000:0.0:1788572333.891391:0:15204:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.202.113@tcp lock: ffff9ada4542aa00/0xef30c340bfb45a40 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1408400000000 nid: local remote: 0xdb7edb52b831ca7a expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00000001:0.0:1788572333.891396:0:15204:0:(mgc_request.c:917:mgc_blocking_ast()) Process entered 10000000:00010000:0.0:1788572333.891399:0:15204:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.202.113@tcp lock: ffff9ada4542aa00/0xef30c340bfb45a40 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1409400000000 nid: local remote: 0xdb7edb52b831ca7a expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1788572333.891405:0:15204:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 10000000:00000001:0.0:1788572333.891408:0:15204:0:(mgc_request.c:719:mgc_requeue_add()) Process entered 10000000:00000040:0.0:1788572333.891410:0:15204:0:(mgc_request.c:723:mgc_requeue_add()) log lustre-cliir: requeue (r=2 sp=0 st=1) 10000000:00000040:0.0:1788572333.891411:0:15204:0:(mgc_request.c:726:mgc_requeue_add()) lockh 0x0 00010000:00000001:0.0:1788572333.891413:0:15204:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00010000:00000001:0.0:1788572333.891414:0:15204:0:(ldlm_lock.c:617:__ldlm_handle2lock()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:0.0:1788572333.891437:0:15204:0:(mgc_request.c:742:mgc_requeue_add()) Process leaving 10000000:00000001:0.0:1788572333.891439:0:15204:0:(mgc_request.c:117:config_log_put()) Process entered 10000000:00000040:0.0:1788572333.891440:0:15204:0:(mgc_request.c:123:config_log_put()) log lustre-cliir(ffff9ada42fa2200) refs 2 10000000:00000001:0.0:1788572333.891443:0:15204:0:(mgc_request.c:147:config_log_put()) Process leaving 10000000:00000001:0.0:1788572333.891444:0:15204:0:(mgc_request.c:967:mgc_blocking_ast()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788572333.891447:0:15204:0:(ldlm_lock.c:2501:ldlm_lock_cancel()) Process entered 00010000:00000001:0.0:1788572333.891451:0:15204:0:(ldlm_lock.c:410:ldlm_lock_destroy_nolock()) Process entered 00010000:00000001:0.0:1788572333.891453:0:15204:0:(ldlm_lock.c:354:ldlm_lock_destroy_internal()) Process entered 00010000:00000001:0.0:1788572333.891455:0:15204:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:0.0:1788572333.891457:0:15204:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:0.0:1788572333.891460:0:15204:0:(lustre_handles.c:107:class_handle_unhash_nolock()) removing object ffff9ada4542aa00 with handle 0xef30c340bfb45a40 from hash 00010000:00000001:0.0:1788572333.891462:0:15204:0:(ldlm_lock.c:384:ldlm_lock_destroy_internal()) Process leaving 00010000:00000001:0.0:1788572333.891464:0:15204:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1788572333.891466:0:15204:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:0.0:1788572333.891467:0:15204:0:(ldlm_lock.c:416:ldlm_lock_destroy_nolock()) Process leaving 00010000:00000001:0.0:1788572333.891469:0:15204:0:(ldlm_lock.c:2537:ldlm_lock_cancel()) Process leaving 00010000:00000001:0.0:1788572333.891471:0:15204:0:(ldlm_request.c:1422:ldlm_cli_cancel_local()) Process leaving (rc=70368744177664 : 70368744177664 : 400000000000) 00010000:00000001:0.0:1788572333.891474:0:15204:0:(ldlm_request.c:2456:ldlm_cli_cancel_list()) Process entered 00010000:00000001:0.0:1788572333.891477:0:15204:0:(ldlm_request.c:1527:ldlm_cli_cancel_req()) Process entered 00000100:00000010:0.0:1788572333.891487:0:15204:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada45462680. 00000020:00000040:0.0:1788572333.891491:0:15204:0:(genops.c:1210:class_import_get()) import ffff9ada58a62800 refcount=4 obd=MGC192.168.202.113@tcp 00000100:00000001:0.0:1788572333.891496:0:15204:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788572333.891498:0:15204:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788572333.891505:0:15204:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788572333.891513:0:15204:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at ffff9ada456bb400. 00000100:00000001:0.0:1788572333.891520:0:15204:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788572333.891523:0:15204:0:(ldlm_request.c:1461:ldlm_cancel_pack()) Process entered 00010000:00010000:0.0:1788572333.891528:0:15204:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.202.113@tcp lock: ffff9ada4542aa00/0xef30c340bfb45a40 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0xdb7edb52b831ca7a expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572333.891534:0:15204:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00000001:0.0:1788572333.891536:0:15204:0:(ldlm_request.c:1500:ldlm_cancel_pack()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572333.891540:0:15204:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788572333.891546:0:15204:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572333.891554:0:15204:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000e513b575] to pc [ptlrpcd_00_00+0] req@ffff9ada45462680 x1875449616092672/t0(0) o103->MGC192.168.202.113@tcp@192.168.202.113@tcp:17/18 lens 328/224 e 0 to 0 dl 0 ref 1 fl New:QU/200/ffffffff rc 0/-1 job:'ldlm_bl.0' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788572333.891578:0:15204:0:(ldlm_request.c:1595:ldlm_cli_cancel_req()) Process leaving via out (rc=0 : 0 : 0x0) 00010000:00000001:0.0:1788572333.891581:0:15204:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1788572333.891582:0:15204:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:0.0:1788572333.891583:0:15204:0:(ldlm_request.c:2504:ldlm_cli_cancel_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788572333.891585:0:15204:0:(ldlm_request.c:1834:ldlm_cli_cancel()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:0.0:1788572333.891587:0:15204:0:(mgc_request.c:967:mgc_blocking_ast()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572333.891590:0:15204:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.202.113@tcp lock: ffff9ada4542aa00/0xef30c340bfb45a40 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0xdb7edb52b831ca7a expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:0.0:1788572333.891596:0:15204:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00010000:0.0:1788572333.891598:0:15204:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.202.113@tcp lock: ffff9ada4542aa00/0xef30c340bfb45a40 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0xdb7edb52b831ca7a expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000040:0.0:1788572333.891606:0:15204:0:(ldlm_resource.c:1704:ldlm_resource_putref()) putref res: ffff9ada6047fc00 count: 0 00010000:00000001:0.0:1788572333.891621:0:15204:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:0.0:1788572333.891623:0:15204:0:(ldlm_lockd.c:1935:ldlm_handle_bl_callback()) Process leaving 00010000:00000010:0.0:1788572333.891625:0:15204:0:(ldlm_lockd.c:2919:ldlm_bl_thread_blwi()) kfreed 'blwi': 192 at ffff9ada4352f6c0. 00010000:00000001:0.0:1788572333.891630:0:15204:0:(ldlm_lockd.c:2923:ldlm_bl_thread_blwi()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572333.891632:0:15204:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572333.891634:0:15204:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572333.891636:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000001:0.0:1788572333.891638:0:15204:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572333.891639:0:15204:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572333.891641:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000001:0.0:1788572333.891642:0:15204:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572333.891643:0:15204:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572333.891645:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000001:0.0:1788572333.891653:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:0.0:1788572333.891657:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00000001:0.0:1788572333.891659:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572333.891665:0:2384:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788572333.891667:0:2384:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572333.891669:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788572333.891671:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788572333.891674:0:2384:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788572333.891681:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada45462680 x1875449616092672/t0(0) o103->MGC192.168.202.113@tcp@192.168.202.113@tcp:17/18 lens 328/224 e 0 to 0 dl 0 ref 1 fl New:QU/200/ffffffff rc 0/-1 job:'ldlm_bl.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572333.891689:0:2384:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788572333.895210:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572333.895224:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572333.895341:0:2384:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572333.895352:0:2384:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788572333.895357:0:2384:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788572333.895360:0:2384:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788572333.895373:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45462680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449616092672:192.168.202.113@tcp:103:ldlm_bl.0 00000100:00000001:0.0:1788572333.895379:0:2384:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788572333.895387:0:2384:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788572333.895390:0:2384:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572333.895393:0:2384:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788572333.895400:0:2384:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada456bb800. 02000000:00000001:0.0:1788572333.895402:0:2384:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788572333.895412:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:0.0:1788572333.895417:0:2384:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada60acaf20. 00000400:00000010:0.0:1788572333.895426:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada5176aaa0. 00000100:00000200:0.0:1788572333.895435:0:2384:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616092672, portal 18 00000100:00000001:0.0:1788572333.895438:0:2384:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788572333.895439:0:2384:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860988614272 : -111212720937344 : ffff9ada45462680) 00000100:00000040:0.0:1788572333.895449:0:2384:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada45462680 x1875449616092672/t0(0) o103->MGC192.168.202.113@tcp@192.168.202.113@tcp:17/18 lens 328/224 e 0 to 0 dl 1788572349 ref 2 fl Rpc:r/200/ffffffff rc 0/-1 job:'ldlm_bl.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572333.895459:0:2384:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788572333.895462:0:2384:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:0.0:1788572333.895465:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada5176a110. 00000100:00000200:0.0:1788572333.895470:0:2384:0:(niobuf.c:94:ptl_send_buf()) Sending 328 bytes to portal 17, xid 1875449616092672, offset 0 00000400:00000200:0.0:1788572333.895476:0:2384:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:0.0:1788572333.895487:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:0.0:1788572333.895496:0:2384:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:0.0:1788572333.895503:0:2384:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788572333.895508:0:2384:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:0.0:1788572333.895513:0:2384:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788572333.895516:0:2384:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:0.0:1788572333.895520:0:2384:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22295:22295:255:4294967295] 192.168.202.113@tcp LPNI seq info [22291:22291:7:4294967295] 00000400:00000200:0.0:1788572333.895526:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:0.0:1788572333.895537:0:2384:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:0.0:1788572333.895542:0:2384:0:(socklnd_cb.c:931:ksocknal_send()) sending 328 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:0.0:1788572333.895548:0:2384:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada5074ad00. 00000800:00000200:0.0:1788572333.895555:0:2384:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:0.0:1788572333.895563:0:2384:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:0.0:1788572333.895569:0:2384:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada5074ad00 type 1, nob 424 niov 1 nkiov 1 00000100:00000001:0.0:1788572333.895573:0:2384:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788572333.895578:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:0.0:1788572333.895580:0:2384:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS new deadline estimation for cpu 2 is 4298929436 00000100:00000001:0.0:1788572333.895589:0:2384:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4298927950 00000100:00000001:0.0:1788572333.895594:0:2384:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS new deadline estimation for cpu 3 is 4298929436 00000100:00000001:0.0:1788572333.895597:0:2384:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4298927950 00000100:00000001:0.0:1788572333.895600:0:2384:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788572333.895604:0:2384:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572333.895609:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572333.895612:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788572333.895668:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788572333.895674:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada50c31a00. 00000400:00000200:2.0:1788572333.895683:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572333.895690:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:2.0:1788572333.895696:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada51269f68 00000400:00000010:2.0:1788572333.895699:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada51269f68. 00000100:00000001:2.0:1788572333.895704:0:2378:0:(events.c:390:reply_out_callback()) Process entered 00000100:00000001:2.0:1788572333.895706:0:2378:0:(events.c:402:reply_out_callback()) Process leaving 00000800:00000200:2.0:1788572333.900686:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788572333.900699:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada5074ad00. 00000400:00000200:2.0:1788572333.900708:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572333.900720:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:2.0:1788572333.900729:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada5176a110 00000400:00000010:2.0:1788572333.900732:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada5176a110. 00000100:00000001:2.0:1788572333.900739:0:2378:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:2.0:1788572333.900749:0:2378:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada45462680 x1875449616092672/t0(0) o103->MGC192.168.202.113@tcp@192.168.202.113@tcp:17/18 lens 328/224 e 0 to 0 dl 1788572349 ref 2 fl Rpc:r/200/ffffffff rc 0/-1 job:'ldlm_bl.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572333.900763:0:2378:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572333.900767:0:2378:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada45462680 x1875449616092672/t0(0) o103->MGC192.168.202.113@tcp@192.168.202.113@tcp:17/18 lens 328/224 e 0 to 0 dl 1788572349 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'ldlm_bl.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572333.900786:0:2378:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572333.900788:0:2378:0:(events.c:69:request_out_callback()) Process leaving 00010000:00000010:0.1F:1788572333.905453:0:0:0:(ldlm_resource.c:1555:__ldlm_resource_free()) slab-freed '(res)': 256 at ffff9ada6047fc00. 00010000:00000010:0.1:1788572333.905497:0:0:0:(ldlm_lock.c:174:lock_handle_free()) slab-freed 'lock': 512 at ffff9ada4542aa00. 00000800:00000001:2.0:1788572333.910352:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572333.910378:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788572333.910384:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572333.910387:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572333.910398:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572333.910407:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:2.0:1788572333.910419:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572333.910425:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572333.910431:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572333.910436:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 18 MB=0x6a9b624294e00 00000400:00000200:2.0:1788572333.910442:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572333.910449:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 12 from 12345-192.168.202.113@tcp of length 224/224 into md 0x357b5 [1] + 224 00000800:00000001:2.0:1788572333.910458:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572333.910471:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572333.910476:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572333.910482:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:2.0:1788572333.910493:0:2377:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:2.0:1788572333.910576:0:2377:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada45462680 x1875449616092672/t0(0) o103->MGC192.168.202.113@tcp@192.168.202.113@tcp:17/18 lens 328/224 e 0 to 0 dl 1788572349 ref 1 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'ldlm_bl.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788572333.910588:0:2377:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada45462680 x1875449616092672/t0(0) o103->MGC192.168.202.113@tcp@192.168.202.113@tcp:17/18 lens 328/224 e 0 to 0 dl 1788572349 ref 1 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'ldlm_bl.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572333.910612:0:2377:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788572333.910624:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572333.910628:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572333.914989:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788572333.915003:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788572333.915017:0:2384:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788572333.915020:0:2384:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788572333.915026:0:2384:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada60acaf20. 00000400:00000200:0.0:1788572333.915040:0:2384:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada5176aaa0 00000400:00000010:0.0:1788572333.915043:0:2384:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada5176aaa0. 00000100:00000001:0.0:1788572333.915049:0:2384:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:0.0:1788572333.915065:0:2384:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada45462680 x1875449616092672/t0(0) o103->MGC192.168.202.113@tcp@192.168.202.113@tcp:17/18 lens 328/224 e 0 to 0 dl 1788572349 ref 1 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'ldlm_bl.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788572333.915079:0:2384:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada45462680 x1875449616092672/t0(0) o103->MGC192.168.202.113@tcp@192.168.202.113@tcp:17/18 lens 328/224 e 0 to 0 dl 1788572349 ref 1 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'ldlm_bl.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572333.915088:0:2384:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788572333.915091:0:2384:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572333.915094:0:2384:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:0.0:1788572333.915098:0:2384:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788572333.915101:0:2384:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788572333.915103:0:2384:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572333.915106:0:2384:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788572333.915115:0:2384:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada58a62ca8 time=991 v=5 (5 0 0 0) 00000100:00001000:0.0:1788572333.915119:0:2384:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada58a62c18 time=125 v=5 (1 0 0 0) 00000100:00000001:0.0:1788572333.915122:0:2384:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788572333.915124:0:2384:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788572333.915128:0:2384:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788572333.915130:0:2384:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572333.915133:0:2384:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572333.915138:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada45462680 x1875449616092672/t0(0) o103->MGC192.168.202.113@tcp@192.168.202.113@tcp:17/18 lens 328/224 e 0 to 0 dl 1788572349 ref 1 fl Rpc:RQU/200/0 rc 0/0 job:'ldlm_bl.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572333.915145:0:2384:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788572333.915149:0:2384:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788572333.915151:0:2384:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788572333.915154:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada45462680 x1875449616092672/t0(0) o103->MGC192.168.202.113@tcp@192.168.202.113@tcp:17/18 lens 328/224 e 0 to 0 dl 1788572349 ref 1 fl Interpret:RQU/200/0 rc 0/0 job:'ldlm_bl.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572333.915163:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45462680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449616092672:192.168.202.113@tcp:103:ldlm_bl.0 00000100:00000001:0.0:1788572333.915168:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572333.915171:0:2384:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788572333.915175:0:2384:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45462680 x1875449616092672/t0(0) o103->MGC192.168.202.113@tcp@192.168.202.113@tcp:17/18 lens 328/224 e 0 to 0 dl 1788572349 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'ldlm_bl.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572333.915182:0:2384:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788572333.915185:0:2384:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788572333.915189:0:2384:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada456bb800. 02000000:00000001:0.0:1788572333.915194:0:2384:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788572333.915198:0:2384:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:0.0:1788572333.915200:0:2384:0:(genops.c:1223:class_import_put()) import ffff9ada58a62800 refcount=3 obd=MGC192.168.202.113@tcp 00000020:00000001:0.0:1788572333.915202:0:2384:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:0.0:1788572333.915205:0:2384:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada456bb400. 02000000:00000001:0.0:1788572333.915209:0:2384:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788572333.915211:0:2384:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788572333.915213:0:2384:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45462680. 00000100:00000001:0.0:1788572333.915217:0:2384:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788572333.915218:0:2384:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572333.915220:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572333.915232:0:2384:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788572333.915233:0:2384:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572333.915235:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788572333.915238:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572333.943155:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572333.943166:0:365:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:2.0:1788572333.943169:0:365:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:2.0:1788572333.943173:0:365:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572333.943179:0:365:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:2.0:1788572333.943182:0:365:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572333.943186:0:365:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572333.943188:0:365:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572333.943192:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572333.943194:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572333.943198:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572333.943199:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572333.943203:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572333.943204:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788572333.943416:0:33423:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788572333.943421:0:33423:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788572333.943424:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572334.967048:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572334.967059:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572334.967067:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572334.967068:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572334.967072:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572334.967073:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572334.967077:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572334.967078:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572334.970482:0:15205:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572334.970491:0:15205:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572334.970498:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:00000001:1.0:1788572335.671043:0:15207:0:(mgc_request.c:105:config_log_get()) Process entered 10000000:00000040:1.0:1788572335.671053:0:15207:0:(mgc_request.c:108:config_log_get()) log lustre-cliir (ffff9ada42fa2200) refs 2 10000000:00000001:1.0:1788572335.671060:0:15207:0:(mgc_request.c:109:config_log_get()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:1.0:1788572335.671062:0:15207:0:(mgc_request.c:117:config_log_put()) Process entered 10000000:00000001:1.0:1788572335.671064:0:15207:0:(mgc_request.c:120:config_log_put()) Process leaving 10000000:00000001:1.0:1788572335.671065:0:15207:0:(mgc_request.c:601:do_requeue()) Process entered 10000000:01000000:1.0:1788572335.671069:0:15207:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:00000001:1.0:1788572335.671071:0:15207:0:(mgc_request.c:1783:mgc_process_log()) Process entered 10000000:01000000:1.0:1788572335.671073:0:15207:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9ada58b0a000 from 208 10000000:00000001:1.0:1788572335.671076:0:15207:0:(mgc_request.c:990:mgc_enqueue()) Process entered 10000000:01000000:1.0:1788572335.671078:0:15207:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00000100:00000010:1.0:1788572335.671093:0:15207:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada456bfb80. 00000020:00000040:1.0:1788572335.671098:0:15207:0:(genops.c:1210:class_import_get()) import ffff9ada58a62800 refcount=4 obd=MGC192.168.202.113@tcp 00000100:00000001:1.0:1788572335.671104:0:15207:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788572335.671108:0:15207:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788572335.671115:0:15207:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788572335.671125:0:15207:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at ffff9ada607e1000. 00000100:00000001:1.0:1788572335.671133:0:15207:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572335.671140:0:15207:0:(ldlm_request.c:1064:ldlm_cli_enqueue()) Process entered 00010000:00000001:1.0:1788572335.671143:0:15207:0:(ldlm_lock.c:1700:ldlm_lock_create()) Process entered 00010000:00000010:1.0:1788572335.671151:0:15207:0:(ldlm_resource.c:1510:ldlm_resource_new()) slab-alloced 'res': 256 at ffff9ada607fee00. 00010000:00000001:1.0:1788572335.671157:0:15207:0:(ldlm_lock.c:433:ldlm_lock_new()) Process entered 00010000:00000010:1.0:1788572335.671161:0:15207:0:(ldlm_lock.c:437:ldlm_lock_new()) slab-alloced 'lock': 512 at ffff9ada607f2000. 00000020:00000001:1.0:1788572335.671167:0:15207:0:(lustre_handles.c:61:class_handle_hash()) Process entered 00000020:00000040:1.0:1788572335.671170:0:15207:0:(lustre_handles.c:93:class_handle_hash()) added object ffff9ada607f2000 with handle 0xef30c340bfb4a74e to hash 00000020:00000001:1.0:1788572335.671173:0:15207:0:(lustre_handles.c:94:class_handle_hash()) Process leaving 00010000:00000001:1.0:1788572335.671174:0:15207:0:(ldlm_lock.c:488:ldlm_lock_new()) Process leaving (rc=18446632861445332992 : -111212264218624 : ffff9ada607f2000) 00010000:00000001:1.0:1788572335.671177:0:15207:0:(ldlm_lock.c:1741:ldlm_lock_create()) Process leaving (rc=18446632861445332992 : -111212264218624 : ffff9ada607f2000) 00010000:00000001:1.0:1788572335.671181:0:15207:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:1.0:1788572335.671186:0:15207:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788572335.671190:0:15207:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.202.113@tcp lock: ffff9ada607f2000/0xef30c340bfb4a74e lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572335.671202:0:15207:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.202.113@tcp lock: ffff9ada607f2000/0xef30c340bfb4a74e lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572335.671216:0:15207:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.202.113@tcp lock: ffff9ada607f2000/0xef30c340bfb4a74e lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00000001:1.0:1788572335.671222:0:15207:0:(client.c:3353:ptlrpc_queue_wait()) Process entered 00000100:00000001:1.0:1788572335.671224:0:15207:0:(client.c:1169:ptlrpc_prep_set()) Process entered 00000100:00000010:1.0:1788572335.671233:0:15207:0:(client.c:1171:ptlrpc_prep_set()) kmalloced '(set)': 144 at ffff9ada60aedc00. 00000100:00000001:1.0:1788572335.671235:0:15207:0:(client.c:1186:ptlrpc_prep_set()) Process leaving (rc=18446632861448461312 : -111212261090304 : ffff9ada60aedc00) 00000100:00000001:1.0:1788572335.671239:0:15207:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788572335.671240:0:15207:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860991093632 : -111212718457984 : ffff9ada456bfb80) 00000100:00000001:1.0:1788572335.671247:0:15207:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788572335.671253:0:15207:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572335.671255:0:15207:0:(client.c:2689:ptlrpc_set_wait()) Process entered 00000100:00000001:1.0:1788572335.671257:0:15207:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788572335.671271:0:15207:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada456bfb80 x1875449616092800/t0(0) o101->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 328/344 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572335.671280:0:15207:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788572335.671283:0:15207:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572335.671287:0:15207:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788572335.671290:0:15207:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788572335.671293:0:15207:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788572335.671297:0:15207:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada456bfb80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449616092800:192.168.202.113@tcp:101:ll_cfg_requeue.0 00000100:00000001:1.0:1788572335.671302:0:15207:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788572335.671309:0:15207:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788572335.671312:0:15207:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572335.671315:0:15207:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788572335.671320:0:15207:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at ffff9ada454bbc00. 02000000:00000001:1.0:1788572335.671322:0:15207:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788572335.671331:0:15207:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:1.0:1788572335.671348:0:15207:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada600d1268. 00000400:00000010:1.0:1788572335.671356:0:15207:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d23b8. 00000100:00000200:1.0:1788572335.671365:0:15207:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1875449616092800, portal 25 00000100:00000001:1.0:1788572335.671367:0:15207:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788572335.671369:0:15207:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860991093632 : -111212718457984 : ffff9ada456bfb80) 00000100:00000040:1.0:1788572335.671375:0:15207:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada456bfb80 x1875449616092800/t0(0) o101->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 328/344 e 0 to 0 dl 1788572351 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572335.671383:0:15207:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788572335.671386:0:15207:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:1.0:1788572335.671389:0:15207:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2b28. 00000100:00000200:1.0:1788572335.671393:0:15207:0:(niobuf.c:94:ptl_send_buf()) Sending 328 bytes to portal 26, xid 1875449616092800, offset 0 00000400:00000200:1.0:1788572335.671401:0:15207:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:1.0:1788572335.671412:0:15207:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572335.671422:0:15207:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:1.0:1788572335.671428:0:15207:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788572335.671433:0:15207:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:1.0:1788572335.671437:0:15207:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788572335.671440:0:15207:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:1.0:1788572335.671444:0:15207:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22296:22296:256:4294967295] 192.168.202.113@tcp LPNI seq info [22292:22292:8:4294967295] 00000400:00000200:1.0:1788572335.671448:0:15207:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572335.671460:0:15207:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:1.0:1788572335.671466:0:15207:0:(socklnd_cb.c:931:ksocknal_send()) sending 328 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:1.0:1788572335.671471:0:15207:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada48e6fc00. 00000800:00000200:1.0:1788572335.671478:0:15207:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:1.0:1788572335.671487:0:15207:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:1.0:1788572335.671492:0:15207:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada48e6fc00 type 1, nob 424 niov 1 nkiov 1 00000100:00000001:1.0:1788572335.671514:0:15207:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788572335.671519:0:15207:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:1.0:1788572335.671521:0:15207:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS new deadline estimation for cpu 2 is 4298931212 00000100:00000001:1.0:1788572335.671529:0:15207:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4298929436 00000100:00000001:1.0:1788572335.671535:0:15207:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS new deadline estimation for cpu 3 is 4298931212 00000100:00000001:1.0:1788572335.671539:0:15207:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4298929436 00000100:00000001:1.0:1788572335.671541:0:15207:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572335.671544:0:15207:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572335.671546:0:15207:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788572335.671547:0:15207:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:1.0:1788572335.671551:0:15207:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60aedc00 going to sleep for 11 seconds 00000100:00000001:1.0:1788572335.671553:0:15207:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788572335.671556:0:15207:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788572335.671763:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788572335.671768:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada48e6fc00. 00000400:00000200:2.0:1788572335.671774:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572335.671779:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:2.0:1788572335.671787:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2b28 00000400:00000010:2.0:1788572335.671790:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2b28. 00000100:00000001:2.0:1788572335.671796:0:2378:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:2.0:1788572335.671800:0:2378:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada456bfb80 x1875449616092800/t0(0) o101->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 328/344 e 0 to 0 dl 1788572351 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572335.671810:0:2378:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572335.671814:0:2378:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 2 req@ffff9ada456bfb80 x1875449616092800/t0(0) o101->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 328/344 e 0 to 0 dl 1788572351 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572335.671836:0:2378:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572335.671838:0:2378:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000001:2.0:1788572335.673917:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572335.673941:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788572335.673948:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572335.673951:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572335.673960:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572335.673969:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:2.0:1788572335.673980:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572335.673986:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572335.673991:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572335.673997:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 344 into portal 25 MB=0x6a9b624294e80 00000400:00000200:2.0:1788572335.674004:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572335.674010:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.202.113@tcp of length 344/344 into md 0x357bd [1] + 224 00000800:00000001:2.0:1788572335.674017:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572335.674025:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572335.674029:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572335.674033:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:2.0:1788572335.674042:0:2377:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:2.0:1788572335.674052:0:2377:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada456bfb80 x1875449616092800/t0(0) o101->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 328/344 e 0 to 0 dl 1788572351 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788572335.674064:0:2377:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=344 offset=224 replen=344 req@ffff9ada456bfb80 x1875449616092800/t0(0) o101->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 328/344 e 0 to 0 dl 1788572351 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572335.674086:0:2377:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788572335.674097:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572335.674101:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572335.674283:0:15207:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788572335.674293:0:15207:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788572335.674295:0:15207:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788572335.674299:0:15207:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada600d1268. 00000400:00000200:1.0:1788572335.674305:0:15207:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d23b8 00000400:00000010:1.0:1788572335.674308:0:15207:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d23b8. 00000100:00000001:1.0:1788572335.674312:0:15207:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:1.0:1788572335.674317:0:15207:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada456bfb80 x1875449616092800/t0(0) o101->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 328/344 e 0 to 0 dl 1788572351 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788572335.674326:0:15207:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada456bfb80 x1875449616092800/t0(0) o101->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 328/344 e 0 to 0 dl 1788572351 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572335.674333:0:15207:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788572335.674348:0:15207:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572335.674352:0:15207:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1788572335.674356:0:15207:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788572335.674359:0:15207:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788572335.674362:0:15207:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572335.674365:0:15207:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788572335.674374:0:15207:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada58a62c60 time=97 v=5 (5 5 5 5) 00000100:00001000:1.0:1788572335.674378:0:15207:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada58a62c18 time=127 v=5 (1 0 0 0) 00000100:00000001:1.0:1788572335.674381:0:15207:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788572335.674383:0:15207:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572335.674387:0:15207:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788572335.674390:0:15207:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572335.674393:0:15207:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572335.674398:0:15207:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada456bfb80 x1875449616092800/t0(0) o101->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 328/344 e 0 to 0 dl 1788572351 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572335.674405:0:15207:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572335.674409:0:15207:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788572335.674411:0:15207:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788572335.674416:0:15207:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada456bfb80 x1875449616092800/t0(0) o101->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 328/344 e 0 to 0 dl 1788572351 ref 2 fl Interpret:RQU/200/0 rc 0/0 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572335.674424:0:15207:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada456bfb80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449616092800:192.168.202.113@tcp:101:ll_cfg_requeue.0 00000100:00000001:1.0:1788572335.674429:0:15207:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572335.674432:0:15207:0:(client.c:2780:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572335.674435:0:15207:0:(client.c:1237:ptlrpc_set_destroy()) Process entered 00000100:00000001:1.0:1788572335.674438:0:15207:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788572335.674442:0:15207:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada456bfb80 x1875449616092800/t0(0) o101->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 328/344 e 0 to 0 dl 1788572351 ref 2 fl Complete:RQU/200/0 rc 0/0 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572335.674449:0:15207:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788572335.674453:0:15207:0:(client.c:3802:ptlrpc_reqset_free()) kfreed 'set': 144 at ffff9ada60aedc00. 00000100:00000001:1.0:1788572335.674457:0:15207:0:(client.c:1274:ptlrpc_set_destroy()) Process leaving 00000100:00000001:1.0:1788572335.674459:0:15207:0:(client.c:3372:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572335.674463:0:15207:0:(ldlm_request.c:661:ldlm_cli_enqueue_fini()) Process entered 00010000:00000001:1.0:1788572335.674467:0:15207:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:1.0:1788572335.674471:0:15207:0:(lustre_handles.c:130:class_handle2object()) Process entered 00000020:00000040:1.0:1788572335.674474:0:15207:0:(lustre_handles.c:148:class_handle2object()) GET ldlm ffff9ada607f2000 refcount=4 00000020:00000001:1.0:1788572335.674477:0:15207:0:(lustre_handles.c:155:class_handle2object()) Process leaving (rc=18446632861445332992 : -111212264218624 : ffff9ada607f2000) 00010000:00000001:1.0:1788572335.674481:0:15207:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446632861445332992 : -111212264218624 : ffff9ada607f2000) 00010000:00000040:1.0:1788572335.674489:0:15207:0:(ldlm_request.c:746:ldlm_cli_enqueue_fini()) local: ffff9ada607f2000, remote cookie: 0xdb7edb52b8334db1, flags: 0x0 00010000:00000001:1.0:1788572335.674493:0:15207:0:(ldlm_lock.c:1795:ldlm_lock_enqueue()) Process entered 00010000:00000001:1.0:1788572335.674498:0:15207:0:(ldlm_lock.c:1125:ldlm_grant_lock()) Process entered 00010000:00000001:1.0:1788572335.674500:0:15207:0:(ldlm_lock.c:989:search_granted_lock()) Process entered 00010000:00000001:1.0:1788572335.674502:0:15207:0:(ldlm_lock.c:1055:search_granted_lock()) Process leaving 00010000:00000001:1.0:1788572335.674504:0:15207:0:(ldlm_lock.c:1067:ldlm_granted_list_add_lock()) Process entered 00010000:00000040:1.0:1788572335.674507:0:15207:0:(ldlm_resource.c:1876:ldlm_resource_dump()) --- Resource: [0x65727473756c:0x2:0x0].0x0 (ffff9ada607fee00) refcount = 1 00010000:00010000:1.0:1788572335.674514:0:15207:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.202.113@tcp lock: ffff9ada607f2000/0xef30c340bfb4a74e lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b8334db1 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:1.0:1788572335.674522:0:15207:0:(ldlm_lock.c:1094:ldlm_granted_list_add_lock()) Process leaving 00010000:00000001:1.0:1788572335.674524:0:15207:0:(ldlm_lock.c:1157:ldlm_grant_lock()) Process leaving 00010000:00000001:1.0:1788572335.674526:0:15207:0:(ldlm_lock.c:1914:ldlm_lock_enqueue()) Process leaving via out (rc=0 : 0 : 0x0) 00010000:00000001:1.0:1788572335.674530:0:15207:0:(ldlm_request.c:286:ldlm_completion_ast()) Process entered 00010000:00000001:1.0:1788572335.674532:0:15207:0:(ldlm_request.c:295:ldlm_completion_ast()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788572335.674536:0:15207:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.202.113@tcp lock: ffff9ada607f2000/0xef30c340bfb4a74e lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0xdb7edb52b8334db1 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:1.0:1788572335.674541:0:15207:0:(ldlm_request.c:842:ldlm_cli_enqueue_fini()) Process leaving 00010000:00000001:1.0:1788572335.674543:0:15207:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788572335.674546:0:15207:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:1.0:1788572335.674547:0:15207:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788572335.674548:0:15207:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:1.0:1788572335.674550:0:15207:0:(ldlm_request.c:1218:ldlm_cli_enqueue()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572335.674553:0:15207:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788572335.674557:0:15207:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada456bfb80 x1875449616092800/t0(0) o101->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 328/344 e 0 to 0 dl 1788572351 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572335.674564:0:15207:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788572335.674567:0:15207:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788572335.674571:0:15207:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada454bbc00. 02000000:00000001:1.0:1788572335.674575:0:15207:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788572335.674578:0:15207:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788572335.674580:0:15207:0:(genops.c:1223:class_import_put()) import ffff9ada58a62800 refcount=3 obd=MGC192.168.202.113@tcp 00000020:00000001:1.0:1788572335.674582:0:15207:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1788572335.674586:0:15207:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada607e1000. 02000000:00000001:1.0:1788572335.674589:0:15207:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788572335.674591:0:15207:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788572335.674593:0:15207:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada456bfb80. 00000100:00000001:1.0:1788572335.674597:0:15207:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788572335.674599:0:15207:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 10000000:00000001:1.0:1788572335.674601:0:15207:0:(mgc_request.c:1023:mgc_enqueue()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:1.0:1788572335.674605:0:15207:0:(mgc_request.c:105:config_log_get()) Process entered 10000000:00000040:1.0:1788572335.674608:0:15207:0:(mgc_request.c:108:config_log_get()) log lustre-cliir (ffff9ada42fa2200) refs 3 10000000:00000001:1.0:1788572335.674610:0:15207:0:(mgc_request.c:109:config_log_get()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572335.674613:0:15207:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:1.0:1788572335.674614:0:15207:0:(lustre_handles.c:130:class_handle2object()) Process entered 00000020:00000040:1.0:1788572335.674616:0:15207:0:(lustre_handles.c:148:class_handle2object()) GET ldlm ffff9ada607f2000 refcount=3 00000020:00000001:1.0:1788572335.674618:0:15207:0:(lustre_handles.c:155:class_handle2object()) Process leaving (rc=18446632861445332992 : -111212264218624 : ffff9ada607f2000) 00010000:00000001:1.0:1788572335.674621:0:15207:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446632861445332992 : -111212264218624 : ffff9ada607f2000) 00010000:00000001:1.0:1788572335.674623:0:15207:0:(ldlm_lock.c:2547:ldlm_lock_set_data()) Process entered 00010000:00000001:1.0:1788572335.674625:0:15207:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788572335.674626:0:15207:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:1.0:1788572335.674627:0:15207:0:(ldlm_lock.c:2556:ldlm_lock_set_data()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:1.0:1788572335.674630:0:15207:0:(mgc_request.c:1520:mgc_process_recover_log()) Process entered 10000000:00000010:1.0:1788572335.674636:0:15207:0:(mgc_request.c:1532:mgc_process_recover_log()) kmalloced '(folios)': 32 at ffff9ada60a60600. 00000100:00000010:1.0:1788572335.674664:0:15207:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada456bfb80. 00000020:00000040:1.0:1788572335.674667:0:15207:0:(genops.c:1210:class_import_get()) import ffff9ada58a62800 refcount=4 obd=MGC192.168.202.113@tcp 00000100:00000001:1.0:1788572335.674671:0:15207:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788572335.674673:0:15207:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788572335.674679:0:15207:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788572335.674684:0:15207:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 512 at ffff9ada607e1000. 00000100:00000001:1.0:1788572335.674691:0:15207:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572335.674696:0:15207:0:(client.c:227:ptlrpc_prep_bulk_imp()) Process entered 00000100:00000010:1.0:1788572335.674700:0:15207:0:(client.c:169:ptlrpc_new_bulk()) kmalloced '(desc)': 984 at ffff9ada454bbc00. 00000100:00000010:1.0:1788572335.674703:0:15207:0:(client.c:174:ptlrpc_new_bulk()) kmalloced '(desc->bd_vec)': 64 at ffff9ada516e9cc0. 00000020:00000040:1.0:1788572335.674706:0:15207:0:(genops.c:1210:class_import_get()) import ffff9ada58a62800 refcount=5 obd=MGC192.168.202.113@tcp 00000100:00000001:1.0:1788572335.674712:0:15207:0:(client.c:3353:ptlrpc_queue_wait()) Process entered 00000100:00000001:1.0:1788572335.674714:0:15207:0:(client.c:1169:ptlrpc_prep_set()) Process entered 00000100:00000010:1.0:1788572335.674720:0:15207:0:(client.c:1171:ptlrpc_prep_set()) kmalloced '(set)': 144 at ffff9ada60aedc00. 00000100:00000001:1.0:1788572335.674722:0:15207:0:(client.c:1186:ptlrpc_prep_set()) Process leaving (rc=18446632861448461312 : -111212261090304 : ffff9ada60aedc00) 00000100:00000001:1.0:1788572335.674725:0:15207:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788572335.674726:0:15207:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860991093632 : -111212718457984 : ffff9ada456bfb80) 00000100:00000001:1.0:1788572335.674731:0:15207:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788572335.674736:0:15207:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572335.674737:0:15207:0:(client.c:2689:ptlrpc_set_wait()) Process entered 00000100:00000001:1.0:1788572335.674739:0:15207:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788572335.674744:0:15207:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada456bfb80 x1875449616092928/t0(0) o256->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 304/240 e 0 to 0 dl 0 ref 2 fl New:QU/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572335.674751:0:15207:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788572335.674753:0:15207:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572335.674758:0:15207:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788572335.674760:0:15207:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788572335.674763:0:15207:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788572335.674766:0:15207:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada456bfb80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449616092928:192.168.202.113@tcp:256:ll_cfg_requeue.0 00000100:00000001:1.0:1788572335.674771:0:15207:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788572335.674776:0:15207:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788572335.674778:0:15207:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572335.674781:0:15207:0:(niobuf.c:346:ptlrpc_register_bulk()) Process entered 00000400:00000200:1.0:1788572335.674788:0:15207:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:1.0:1788572335.674791:0:15207:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada600d1268. 00000400:00000010:1.0:1788572335.674799:0:15207:0:(lib-md.c:137:lnet_md_build()) alloc '(lmd)': 184 at ffff9ada60aed3c0. 00000100:00000200:1.0:1788572335.674806:0:15207:0:(niobuf.c:453:ptlrpc_register_bulk()) Setup 1 bulk put-sink buffers: 4 pages 16384 bytes, mbits x0x6a9b624294f00-0x6a9b624294f00, portal 33 00000100:00000001:1.0:1788572335.674809:0:15207:0:(niobuf.c:455:ptlrpc_register_bulk()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572335.674812:0:15207:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788572335.674816:0:15207:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada607e1a00. 02000000:00000001:1.0:1788572335.674833:0:15207:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788572335.674837:0:15207:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:1.0:1788572335.674840:0:15207:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada600d1ec8. 00000400:00000010:1.0:1788572335.674843:0:15207:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d23b8. 00000100:00000200:1.0:1788572335.674847:0:15207:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616092928, portal 25 00000100:00000001:1.0:1788572335.674849:0:15207:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788572335.674850:0:15207:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860991093632 : -111212718457984 : ffff9ada456bfb80) 00000100:00000040:1.0:1788572335.674887:0:15207:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada456bfb80 x1875449616092928/t0(0) o256->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 304/240 e 0 to 0 dl 1788572351 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572335.674895:0:15207:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788572335.674898:0:15207:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:1.0:1788572335.674901:0:15207:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2220. 00000100:00000200:1.0:1788572335.674905:0:15207:0:(niobuf.c:94:ptl_send_buf()) Sending 304 bytes to portal 26, xid 1875449616092928, offset 0 00000400:00000200:1.0:1788572335.674912:0:15207:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:1.0:1788572335.674920:0:15207:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572335.674927:0:15207:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:1.0:1788572335.674933:0:15207:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788572335.674937:0:15207:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:1.0:1788572335.674941:0:15207:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788572335.674944:0:15207:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:1.0:1788572335.674948:0:15207:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22297:22297:256:4294967295] 192.168.202.113@tcp LPNI seq info [22293:22293:8:4294967295] 00000400:00000200:1.0:1788572335.674953:0:15207:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572335.674962:0:15207:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:1.0:1788572335.674968:0:15207:0:(socklnd_cb.c:931:ksocknal_send()) sending 304 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:1.0:1788572335.674973:0:15207:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada48e6f900. 00000800:00000200:1.0:1788572335.674980:0:15207:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:1.0:1788572335.674989:0:15207:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:1.0:1788572335.674994:0:15207:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada48e6f900 type 1, nob 400 niov 1 nkiov 1 00000100:00000001:1.0:1788572335.675012:0:15207:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788572335.675016:0:15207:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:1.0:1788572335.675019:0:15207:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS new deadline estimation for cpu 2 is 4298931216 00000100:00000001:1.0:1788572335.675027:0:15207:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4298931212 00000100:00000001:1.0:1788572335.675032:0:15207:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS new deadline estimation for cpu 3 is 4298931216 00000100:00000001:1.0:1788572335.675037:0:15207:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4298931212 00000100:00000001:1.0:1788572335.675039:0:15207:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572335.675042:0:15207:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572335.675044:0:15207:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788572335.675046:0:15207:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00100000:1.0:1788572335.675048:0:15207:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60aedc00 going to sleep for 11 seconds 00000100:00000001:1.0:1788572335.675050:0:15207:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788572335.675052:0:15207:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788572335.675416:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788572335.675422:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada48e6f900. 00000400:00000200:2.0:1788572335.675427:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572335.675432:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:2.0:1788572335.675438:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2220 00000400:00000010:2.0:1788572335.675440:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2220. 00000100:00000001:2.0:1788572335.675444:0:2378:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:2.0:1788572335.675448:0:2378:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada456bfb80 x1875449616092928/t0(0) o256->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 304/240 e 0 to 0 dl 1788572351 ref 3 fl Rpc:r/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572335.675463:0:2378:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572335.675466:0:2378:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 2 req@ffff9ada456bfb80 x1875449616092928/t0(0) o256->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 304/240 e 0 to 0 dl 1788572351 ref 3 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572335.675471:0:2378:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572335.675473:0:2378:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000001:2.0:1788572335.678844:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572335.678919:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788572335.678926:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572335.678928:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572335.678939:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572335.678949:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:2.0:1788572335.678961:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572335.678966:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572335.678972:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572335.678977:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 44 into portal 33 MB=0x6a9b624294f00 00000400:00000200:2.0:1788572335.678982:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572335.678988:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 21 from 12345-192.168.202.113@tcp of length 44/44 into md 0x357c5 [4] + 0 00000400:00000010:2.0:1788572335.678994:0:2377:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada600d1268. 00000400:00000200:2.0:1788572335.678999:0:2377:0:(lib-md.c:42:lnet_md_unlink()) Queueing unlink of md ffff9ada60aed3c0 00000800:00000001:2.0:1788572335.679005:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572335.679013:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572335.679017:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572335.679021:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:2.0:1788572335.679029:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada60aed3c0 00000400:00000010:2.0:1788572335.679031:0:2377:0:(lib-lnet.h:286:lnet_md_free()) kfreed 'md': 184 at ffff9ada60aed3c0. 00000100:00000001:2.0:1788572335.679037:0:2377:0:(events.c:177:client_bulk_callback()) Process entered 00000100:00000200:2.0:1788572335.679039:0:2377:0:(events.c:195:client_bulk_callback()) event type 2, status 0, req ffff9ada456bfb80 desc ffff9ada454bbc00 mbits 1875449616092928 00000100:00000001:2.0:1788572335.679056:0:2377:0:(events.c:220:client_bulk_callback()) Process leaving 00000400:00000200:2.0:1788572335.679064:0:2377:0:(lib-md.c:96:lnet_cpt_of_md()) Couldn't resolve first page of md 0000000000000000 with offset 0 00000400:00000200:2.0:1788572335.679067:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572335.679073:0:2377:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source Specified: 192.168.202.13@tcp to MR: 192.168.202.113@tcp local destination 00000400:00000200:2.0:1788572335.679080:0:2377:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22298:22298:256:4294967295] 192.168.202.113@tcp LPNI seq info [22294:22294:8:4294967295] 00000400:00000200:2.0:1788572335.679085:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572335.679093:0:2377:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:192.168.202.13@tcp) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) 192.168.202.113@tcp : ACK try# 0 00000800:00000200:2.0:1788572335.679098:0:2377:0:(socklnd_cb.c:931:ksocknal_send()) sending 0 bytes in 4 frags to 12345-192.168.202.113@tcp 00000800:00000010:2.0:1788572335.679105:0:2377:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 296 at ffff9ada45574a00. 00000800:00000200:2.0:1788572335.679111:0:2377:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:2.0:1788572335.679121:0:2377:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:2.0:1788572335.679127:0:2377:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada45574a00 type 0, nob 96 niov 1 nkiov 0 00000800:00000200:2.0:1788572335.679245:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788572335.679250:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 296 at ffff9ada45574a00. 00000400:00000200:2.0:1788572335.679253:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572335.679257:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: ACK: OK 00000800:00000001:2.0:1788572335.679265:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572335.679269:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572335.682005:0:15207:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788572335.682018:0:15207:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788572335.690627:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572335.690660:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788572335.690665:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572335.690667:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572335.690679:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572335.690689:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:2.0:1788572335.690704:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572335.690710:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572335.690718:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572335.690724:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 240 into portal 25 MB=0x6a9b624294f00 00000400:00000200:2.0:1788572335.690730:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572335.690736:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.202.113@tcp of length 240/240 into md 0x357cd [1] + 224 00000800:00000001:2.0:1788572335.690746:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572335.690780:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572335.690784:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572335.690789:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:2.0:1788572335.690802:0:2377:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:2.0:1788572335.690817:0:2377:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada456bfb80 x1875449616092928/t0(0) o256->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 304/240 e 0 to 0 dl 1788572351 ref 2 fl Rpc:Qr/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788572335.690848:0:2377:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=240 offset=224 replen=240 req@ffff9ada456bfb80 x1875449616092928/t0(0) o256->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 304/240 e 0 to 0 dl 1788572351 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572335.690932:0:2377:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788572335.690943:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572335.690947:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572335.691134:0:15207:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788572335.691145:0:15207:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788572335.691148:0:15207:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788572335.691152:0:15207:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada600d1ec8. 00000400:00000200:1.0:1788572335.691158:0:15207:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d23b8 00000400:00000010:1.0:1788572335.691161:0:15207:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d23b8. 00000100:00000001:1.0:1788572335.691165:0:15207:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:1.0:1788572335.691171:0:15207:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada456bfb80 x1875449616092928/t0(0) o256->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 304/240 e 0 to 0 dl 1788572351 ref 2 fl Rpc:RQ/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788572335.691180:0:15207:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada456bfb80 x1875449616092928/t0(0) o256->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 304/240 e 0 to 0 dl 1788572351 ref 2 fl Rpc:RQU/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572335.691187:0:15207:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788572335.691189:0:15207:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572335.691191:0:15207:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1788572335.691195:0:15207:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788572335.691198:0:15207:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788572335.691200:0:15207:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572335.691203:0:15207:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788572335.691211:0:15207:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada58a62c60 time=97 v=5 (5 5 5 5) 00000100:00001000:1.0:1788572335.691215:0:15207:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada58a62c18 time=127 v=5 (1 0 0 0) 00000100:00000001:1.0:1788572335.691218:0:15207:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788572335.691219:0:15207:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572335.691223:0:15207:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788572335.691225:0:15207:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572335.691228:0:15207:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572335.691233:0:15207:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Bulk req@ffff9ada456bfb80 x1875449616092928/t0(0) o256->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 304/240 e 0 to 0 dl 1788572351 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788572335.691242:0:15207:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Bulk to Interpret req@ffff9ada456bfb80 x1875449616092928/t0(0) o256->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 304/240 e 0 to 0 dl 1788572351 ref 2 fl Bulk:RQU/200/0 rc 0/0 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572335.691250:0:15207:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788572335.691253:0:15207:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788572335.691257:0:15207:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada456bfb80 x1875449616092928/t0(0) o256->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 304/240 e 0 to 0 dl 1788572351 ref 2 fl Interpret:RQU/200/0 rc 0/0 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572335.691265:0:15207:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada456bfb80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449616092928:192.168.202.113@tcp:256:ll_cfg_requeue.0 00000100:00000001:1.0:1788572335.691271:0:15207:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572335.691274:0:15207:0:(client.c:2780:ptlrpc_set_wait()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572335.691277:0:15207:0:(client.c:1237:ptlrpc_set_destroy()) Process entered 00000100:00000001:1.0:1788572335.691279:0:15207:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788572335.691282:0:15207:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada456bfb80 x1875449616092928/t0(0) o256->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 304/240 e 0 to 0 dl 1788572351 ref 2 fl Complete:RQU/200/0 rc 0/0 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572335.691289:0:15207:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788572335.691292:0:15207:0:(client.c:3802:ptlrpc_reqset_free()) kfreed 'set': 144 at ffff9ada60aedc00. 00000100:00000001:1.0:1788572335.691296:0:15207:0:(client.c:1274:ptlrpc_set_destroy()) Process leaving 00000100:00000001:1.0:1788572335.691297:0:15207:0:(client.c:3372:ptlrpc_queue_wait()) Process leaving (rc=0 : 0 : 0) 10000000:00000040:1.0:1788572335.691305:0:15207:0:(mgc_request.c:1600:mgc_process_recover_log()) Latest version 208, more 0. 10000000:00000001:1.0:1788572335.691309:0:15207:0:(mgc_request.c:1308:mgc_apply_recover_logs()) Process entered 10000000:00000010:1.0:1788572335.691316:0:15207:0:(mgc_request.c:1331:mgc_apply_recover_logs()) kmalloced '(nidlist)': 640 at ffff9ada454bac00. 10000000:00000010:1.0:1788572335.691320:0:15207:0:(mgc_request.c:1343:mgc_apply_recover_logs()) kmalloced '(obdname)': 128 at ffff9ada6051ba80. 10000000:00000200:1.0:1788572335.691327:0:15207:0:(mgc_request.c:1241:mgc_build_nidlist()) use 1 from 1 NIDs 10000000:00000040:1.0:1788572335.691331:0:15207:0:(mgc_request.c:1440:mgc_apply_recover_logs()) Looking for OBD lustre-OST0000-osc-ffff9ada58b0a000 00010000:00000001:1.0:1788572335.691351:0:15207:0:(ldlm_lib.c:159:client_import_add_nids_to_conn()) Process entered 00000100:00000001:1.0:1788572335.691356:0:15207:0:(lustre_peer.c:229:class_check_uuid()) Process entered 00000100:00000040:1.0:1788572335.691361:0:15207:0:(lustre_peer.c:232:class_check_uuid()) check if uuid 192.168.202.113@tcp has 192.168.202.113@tcp. 00000100:00000001:1.0:1788572335.691364:0:15207:0:(lustre_peer.c:251:class_check_uuid()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572335.691367:0:15207:0:(lustre_peer.c:189:class_add_nids_to_uuid()) Process entered 00000100:00000200:1.0:1788572335.691369:0:15207:0:(lustre_peer.c:194:class_add_nids_to_uuid()) Comparing 192.168.202.113@tcp with 192.168.202.113@tcp 00000100:00000200:1.0:1788572335.691372:0:15207:0:(lustre_peer.c:201:class_add_nids_to_uuid()) Updating UUID '192.168.202.113@tcp' 00000400:00000200:1.0:1788572335.691376:0:15207:0:(peer.c:1448:LNetAddPeer()) Add peer with 1 NIDs 00000400:00001000:1.0:1788572335.691381:0:15207:0:(api-ni.c:3336:LNetNIInit()) refs 1 00000400:00000200:1.0:1788572335.691385:0:15207:0:(peer.c:1462:LNetAddPeer()) Adding NID[0]: 192.168.202.113@tcp 00000400:00000200:1.0:1788572335.691389:0:15207:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572335.691395:0:15207:0:(peer.c:1911:lnet_peer_add()) peer 192.168.202.113@tcp NID flags 0x100001: -17 00000400:00000200:1.0:1788572335.691399:0:15207:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572335.691408:0:15207:0:(peer.c:2979:lnet_discover_peer_locked()) non-blocking discovery 00000400:00000200:1.0:1788572335.691411:0:15207:0:(peer.c:2988:lnet_discover_peer_locked()) peer 192.168.202.113@tcp NID 192.168.202.113@tcp: 0. pending discovery 00000400:00000200:1.0:1788572335.691414:0:15207:0:(peer.c:1516:LNetAddPeer()) A peer exists for NID 192.168.202.113@tcp 00000100:00000001:1.0:1788572335.691417:0:15207:0:(lustre_peer.c:219:class_add_nids_to_uuid()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572335.691419:0:15207:0:(ldlm_lib.c:174:client_import_add_nids_to_conn()) Process leaving (rc=0 : 0 : 0) 10000000:00000040:1.0:1788572335.691422:0:15207:0:(mgc_request.c:1470:mgc_apply_recover_logs()) IR: lustre-OST0000-osc-ffff9ada58b0a000 target is obsoleted(2/3), reconnect 00000100:00000001:1.0:1788572335.691426:0:15207:0:(recover.c:297:ptlrpc_recover_import()) Process entered 00000100:00080000:1.0:1788572335.691429:0:15207:0:(import.c:214:ptlrpc_set_import_discon()) osc: import ffff9ada48407800 already not connected (conn 11, was 0): DISCONN 00010000:00000001:1.0:1788572335.691435:0:15207:0:(ldlm_lib.c:49:import_set_conn()) Process entered 00000100:00000200:1.0:1788572335.691444:0:15207:0:(events.c:564:ptlrpc_uuid_to_peer()) 192.168.202.113@tcp->12345-192.168.202.113@tcp 00000100:00000001:1.0:1788572335.691447:0:15207:0:(connection.c:105:ptlrpc_connection_get()) Process entered 00000400:00000200:1.0:1788572335.691450:0:15207:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572335.691453:0:15207:0:(peer.c:1635:LNetPrimaryNID()) NID 192.168.202.113@tcp primary NID 192.168.202.113@tcp rc 0 00000100:00000001:1.0:1788572335.691457:0:15207:0:(connection.c:157:ptlrpc_connection_addref()) Process entered 00000100:00000040:1.0:1788572335.691459:0:15207:0:(connection.c:162:ptlrpc_connection_addref()) conn=ffff9ada51db1200 refcount 16 to 192.168.202.113@tcp 00000100:00000001:1.0:1788572335.691462:0:15207:0:(connection.c:164:ptlrpc_connection_addref()) Process leaving (rc=18446632861199700480 : -111212509851136 : ffff9ada51db1200) 00000100:00000001:1.0:1788572335.691465:0:15207:0:(connection.c:111:ptlrpc_connection_get()) Process leaving via out (rc=18446632861199700480 : -111212509851136 : 0xffff9ada51db1200) 00000100:00000040:1.0:1788572335.691468:0:15207:0:(connection.c:150:ptlrpc_connection_get()) conn=ffff9ada51db1200 refcount 16 to 192.168.202.113@tcp 00000100:00000040:1.0:1788572335.691471:0:15207:0:(client.c:137:ptlrpc_uuid_to_connection()) 192.168.202.113@tcp -> ffff9ada51db1200 00010000:00080000:1.0:1788572335.691475:0:15207:0:(ldlm_lib.c:85:import_set_conn()) imp ffff9ada48407800@lustre-OST0000-osc-ffff9ada58b0a000: found existing conn 192.168.202.113@tcp, moved to head 00010000:00000001:1.0:1788572335.691478:0:15207:0:(ldlm_lib.c:87:import_set_conn()) Process leaving via out_free (rc=0 : 0 : 0x0) 00010000:00000040:1.0:1788572335.691481:0:15207:0:(lustre_net.h:1977:ptlrpc_connection_put()) PUT conn=ffff9ada51db1200 refcount 15 to 192.168.202.113@tcp 00010000:00000001:1.0:1788572335.691484:0:15207:0:(ldlm_lib.c:114:import_set_conn()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572335.691487:0:15207:0:(import.c:745:ptlrpc_connect_import_locked()) Process entered 00000100:00080000:1.0:1788572335.691490:0:15207:0:(import.c:65:import_set_state_nolock()) ffff9ada48407800 lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00000001:1.0:1788572335.691493:0:15207:0:(import.c:520:import_select_connection()) Process entered 00000100:00080000:1.0:1788572335.691496:0:15207:0:(import.c:545:import_select_connection()) lustre-OST0000-osc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 0 00000400:00000200:1.0:1788572335.691499:0:15207:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572335.691502:0:15207:0:(peer.c:1689:LNetPeerDiscovered()) Peer NID 192.168.202.113@tcp is discovered: rc = 1 00000100:00000040:1.0:1788572335.691506:0:15207:0:(lustre_net.h:1977:ptlrpc_connection_put()) PUT conn=ffff9ada51db1200 refcount 14 to 192.168.202.113@tcp 00000100:00000001:1.0:1788572335.691507:0:15207:0:(connection.c:157:ptlrpc_connection_addref()) Process entered 00000100:00000040:1.0:1788572335.691509:0:15207:0:(connection.c:162:ptlrpc_connection_addref()) conn=ffff9ada51db1200 refcount 15 to 192.168.202.113@tcp 00000100:00000001:1.0:1788572335.691511:0:15207:0:(connection.c:164:ptlrpc_connection_addref()) Process leaving (rc=18446632861199700480 : -111212509851136 : ffff9ada51db1200) 00000020:00000001:1.0:1788572335.691514:0:15207:0:(genops.c:792:class_conn2export()) Process entered 00000020:00000040:1.0:1788572335.691516:0:15207:0:(genops.c:804:class_conn2export()) looking for export cookie 0xef30c340bfb39c83 00000020:00000001:1.0:1788572335.691519:0:15207:0:(lustre_handles.c:130:class_handle2object()) Process entered 00000020:00000040:1.0:1788572335.691522:0:15207:0:(lustre_handles.c:148:class_handle2object()) GET export ffff9ada50c3b800 refcount=4 00000020:00000001:1.0:1788572335.691524:0:15207:0:(lustre_handles.c:155:class_handle2object()) Process leaving (rc=18446632861181392896 : -111212528158720 : ffff9ada50c3b800) 00000020:00000001:1.0:1788572335.691526:0:15207:0:(genops.c:806:class_conn2export()) Process leaving (rc=18446632861181392896 : -111212528158720 : ffff9ada50c3b800) 00000100:00000040:1.0:1788572335.691529:0:15207:0:(lustre_net.h:1977:ptlrpc_connection_put()) PUT conn=ffff9ada51db1200 refcount 14 to 192.168.202.113@tcp 00000100:00000001:1.0:1788572335.691531:0:15207:0:(connection.c:157:ptlrpc_connection_addref()) Process entered 00000100:00000040:1.0:1788572335.691533:0:15207:0:(connection.c:162:ptlrpc_connection_addref()) conn=ffff9ada51db1200 refcount 15 to 192.168.202.113@tcp 00000100:00000001:1.0:1788572335.691535:0:15207:0:(connection.c:164:ptlrpc_connection_addref()) Process leaving (rc=18446632861199700480 : -111212509851136 : ffff9ada51db1200) 00000020:00000040:1.0:1788572335.691538:0:15207:0:(genops.c:879:class_export_put()) PUTting export ffff9ada50c3b800 : new refcount 3 00000100:00080000:1.0:1788572335.691541:0:15207:0:(import.c:665:import_select_connection()) lustre-OST0000-osc-ffff9ada58b0a000: import ffff9ada48407800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00000001:1.0:1788572335.691544:0:15207:0:(import.c:672:import_select_connection()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572335.691547:0:15207:0:(sec.c:1579:sptlrpc_import_sec_adapt()) Process entered 02000000:00000001:1.0:1788572335.691553:0:15207:0:(sec.c:1619:sptlrpc_import_sec_adapt()) Process leaving via out (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788572335.691556:0:15207:0:(sec.c:1647:sptlrpc_import_sec_adapt()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572335.691559:0:15207:0:(obd_class.h:888:obd_reconnect()) Process entered 00000008:00100000:1.0:1788572335.691564:0:15207:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683394 ocd_grant: 8388608, lost: 0. 00000008:00000001:1.0:1788572335.691567:0:15207:0:(osc_request.c:3833:osc_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572335.691568:0:15207:0:(obd_class.h:902:obd_reconnect()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788572335.691574:0:15207:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada456be300. 00000020:00000040:1.0:1788572335.691578:0:15207:0:(genops.c:1210:class_import_get()) import ffff9ada48407800 refcount=3 obd=lustre-OST0000-osc-ffff9ada58b0a000 02000000:00000001:1.0:1788572335.691582:0:15207:0:(sec.c:2038:sptlrpc_sepol_get()) Process entered 02000000:00000001:1.0:1788572335.691584:0:15207:0:(sec.c:2048:sptlrpc_sepol_get()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572335.691587:0:15207:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788572335.691590:0:15207:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788572335.691594:0:15207:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788572335.691602:0:15207:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 1024 at ffff9ada454bb400. 00000100:00000001:1.0:1788572335.691608:0:15207:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788572335.691615:0:15207:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada456be300 x1875449616093056/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788572335.691624:0:15207:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788572335.691629:0:15207:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572335.691634:0:15207:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000004bc8f44d] to pc [ptlrpcd_rcv+-1] req@ffff9ada456be300 x1875449616093056/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572335.691659:0:15207:0:(import.c:877:ptlrpc_connect_import_locked()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572335.691662:0:15207:0:(recover.c:350:ptlrpc_recover_import()) Process leaving 10000000:00000040:1.0:1788572335.691664:0:15207:0:(mgc_request.c:1487:mgc_apply_recover_logs()) ir apply logs 208/208 for lustre-OST0000-osc-ffff9ada58b0a000 10000000:00000010:1.0:1788572335.691667:0:15207:0:(mgc_request.c:1492:mgc_apply_recover_logs()) kfreed 'obdname': 128 at ffff9ada6051ba80. 10000000:00000010:1.0:1788572335.691672:0:15207:0:(mgc_request.c:1494:mgc_apply_recover_logs()) kfreed 'nidlist': 640 at ffff9ada454bac00. 10000000:00000001:1.0:1788572335.691675:0:15207:0:(mgc_request.c:1496:mgc_apply_recover_logs()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572335.691677:0:15207:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788572335.691681:0:15207:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada456bfb80 x1875449616092928/t0(0) o256->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 304/240 e 0 to 0 dl 1788572351 ref 1 fl Complete:RQU/200/0 rc 0/0 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572335.691688:0:15207:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788572335.691690:0:15207:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788572335.691693:0:15207:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada607e1a00. 02000000:00000001:1.0:1788572335.691696:0:15207:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788572335.691698:0:15207:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788572335.691699:0:15207:0:(genops.c:1223:class_import_put()) import ffff9ada58a62800 refcount=4 obd=MGC192.168.202.113@tcp 00000020:00000001:1.0:1788572335.691701:0:15207:0:(genops.c:1230:class_import_put()) Process leaving 00000100:00000001:1.0:1788572335.691703:0:15207:0:(client.c:325:ptlrpc_free_bulk()) Process entered 00000020:00000001:1.0:1788572335.691705:0:15207:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788572335.691706:0:15207:0:(genops.c:1223:class_import_put()) import ffff9ada58a62800 refcount=3 obd=MGC192.168.202.113@tcp 00000020:00000001:1.0:1788572335.691708:0:15207:0:(genops.c:1230:class_import_put()) Process leaving 00000100:00000010:1.0:1788572335.691711:0:15207:0:(client.c:346:ptlrpc_free_bulk()) kvfree 'desc->bd_vec': 64 at ffff9ada516e9cc0. 00000100:00000010:1.0:1788572335.691715:0:15207:0:(client.c:347:ptlrpc_free_bulk()) kfreed 'desc': 984 at ffff9ada454bbc00. 00000100:00000001:1.0:1788572335.691718:0:15207:0:(client.c:348:ptlrpc_free_bulk()) Process leaving 02000000:00000010:1.0:1788572335.691721:0:15207:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 512 at ffff9ada607e1000. 02000000:00000001:1.0:1788572335.691724:0:15207:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788572335.691726:0:15207:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788572335.691727:0:15207:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada456bfb80. 00000100:00000001:1.0:1788572335.691730:0:15207:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788572335.691732:0:15207:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 10000000:00000010:1.0:1788572335.691752:0:15207:0:(mgc_request.c:1652:mgc_process_recover_log()) kvfree 'folios': 32 at ffff9ada60a60600. 10000000:01000000:1.0:1788572335.691757:0:15207:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.202.113@tcp: configuration from log 'lustre-cliir' succeeded (0). 10000000:00000001:1.0:1788572335.691760:0:15207:0:(mgc_request.c:1029:mgc_cancel()) Process entered 00010000:00000001:1.0:1788572335.691763:0:15207:0:(ldlm_lock.c:612:__ldlm_handle2lock()) Process entered 00000020:00000001:1.0:1788572335.691764:0:15207:0:(lustre_handles.c:130:class_handle2object()) Process entered 00000020:00000040:1.0:1788572335.691767:0:15207:0:(lustre_handles.c:148:class_handle2object()) GET ldlm ffff9ada607f2000 refcount=3 00000020:00000001:1.0:1788572335.691769:0:15207:0:(lustre_handles.c:155:class_handle2object()) Process leaving (rc=18446632861445332992 : -111212264218624 : ffff9ada607f2000) 00010000:00000001:1.0:1788572335.691772:0:15207:0:(ldlm_lock.c:634:__ldlm_handle2lock()) Process leaving (rc=18446632861445332992 : -111212264218624 : ffff9ada607f2000) 00010000:00000001:1.0:1788572335.691775:0:15207:0:(ldlm_lock.c:854:ldlm_lock_decref_internal()) Process entered 00010000:00010000:1.0:1788572335.691781:0:15207:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.202.113@tcp lock: ffff9ada607f2000/0xef30c340bfb4a74e lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b8334db1 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:1.0:1788572335.691790:0:15207:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788572335.691792:0:15207:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00010000:1.0:1788572335.691794:0:15207:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.202.113@tcp lock: ffff9ada607f2000/0xef30c340bfb4a74e lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b8334db1 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00000001:1.0:1788572335.691813:0:15207:0:(ldlm_lock.c:923:ldlm_lock_decref_internal()) Process leaving 00010000:00000001:1.0:1788572335.691814:0:15207:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1788572335.691815:0:15207:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 10000000:00000001:1.0:1788572335.691816:0:15207:0:(mgc_request.c:1033:mgc_cancel()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:1.0:1788572335.691818:0:15207:0:(mgc_request.c:1907:mgc_process_log()) Process leaving (rc=0 : 0 : 0) 10000000:00000001:1.0:1788572335.691920:0:15207:0:(mgc_request.c:621:do_requeue()) Process leaving 10000000:00000001:1.0:1788572335.691923:0:15207:0:(mgc_request.c:117:config_log_put()) Process entered 10000000:00000040:1.0:1788572335.691925:0:15207:0:(mgc_request.c:123:config_log_put()) log lustre-cliir(ffff9ada42fa2200) refs 3 10000000:00000001:1.0:1788572335.691927:0:15207:0:(mgc_request.c:147:config_log_put()) Process leaving 00000100:00000001:2.0:1788572335.693110:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788572335.693120:0:2382:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788572335.693123:0:2382:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788572335.693139:0:2382:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada456be300 x1875449616093056/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572335.693153:0:2382:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788572335.693156:0:2382:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572335.693161:0:2382:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788572335.693165:0:2382:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788572335.693168:0:2382:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788572335.693172:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada456be300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616093056:192.168.202.113@tcp:8:ll_cfg_requeue.0 00000100:00000001:2.0:1788572335.693177:0:2382:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788572335.693183:0:2382:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788572335.693186:0:2382:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572335.693189:0:2382:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788572335.693200:0:2382:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 1024 at ffff9ada6056ac00. 02000000:00000001:2.0:1788572335.693203:0:2382:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788572335.693212:0:2382:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:2.0:1788572335.693218:0:2382:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada43d96a50. 00000400:00000010:2.0:1788572335.693228:0:2382:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada43c8faa0. 00000100:00000200:2.0:1788572335.693237:0:2382:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 1024 bytes, xid 1875449616093056, portal 4 00000100:00000001:2.0:1788572335.693240:0:2382:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788572335.693242:0:2382:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860991087360 : -111212718464256 : ffff9ada456be300) 00000100:00000040:2.0:1788572335.693252:0:2382:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada456be300 x1875449616093056/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572345 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572335.693296:0:2382:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788572335.693301:0:2382:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:2.0:1788572335.693305:0:2382:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada43c8f3b8. 00000100:00000200:2.0:1788572335.693309:0:2382:0:(niobuf.c:94:ptl_send_buf()) Sending 520 bytes to portal 28, xid 1875449616093056, offset 0 00000400:00000200:2.0:1788572335.693316:0:2382:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:2.0:1788572335.693327:0:2382:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572335.693336:0:2382:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:2.0:1788572335.693342:0:2382:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788572335.693347:0:2382:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:2.0:1788572335.693351:0:2382:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788572335.693354:0:2382:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:2.0:1788572335.693359:0:2382:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22299:22299:256:4294967295] 192.168.202.113@tcp LPNI seq info [22295:22295:8:4294967295] 00000400:00000200:2.0:1788572335.693364:0:2382:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572335.693374:0:2382:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:2.0:1788572335.693380:0:2382:0:(socklnd_cb.c:931:ksocknal_send()) sending 520 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:2.0:1788572335.693386:0:2382:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada509ed800. 00000800:00000200:2.0:1788572335.693393:0:2382:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:2.0:1788572335.693401:0:2382:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:2.0:1788572335.693406:0:2382:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada509ed800 type 1, nob 616 niov 1 nkiov 1 00000100:00000001:2.0:1788572335.693428:0:2382:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572335.693433:0:2382:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: using a timeout of 18887 usecs (19 jiffies) 00000400:00000200:2.0:1788572335.693436:0:2382:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:2.0:1788572335.693440:0:2382:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298920681 00000100:00000001:2.0:1788572335.693444:0:2382:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298931216 00000100:00000001:2.0:1788572335.693446:0:2382:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298920681 00000100:00000001:2.0:1788572335.693449:0:2382:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298931216 00000100:00000001:2.0:1788572335.693451:0:2382:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788572335.693454:0:2382:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572335.693459:0:2382:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572335.693462:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572335.693471:0:2382:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788572335.693474:0:2382:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=5 : 5 : 5) 00000100:00000001:2.0:1788572335.693476:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788572335.693477:0:2382:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788572335.693479:0:2382:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572335.693481:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788572335.693992:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788572335.693998:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada509ed800. 00000400:00000200:3.0:1788572335.694006:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572335.694014:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:3.0:1788572335.694022:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada43c8f3b8 00000400:00000010:3.0:1788572335.694025:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada43c8f3b8. 00000100:00000001:3.0:1788572335.694029:0:2378:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:3.0:1788572335.694034:0:2378:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada456be300 x1875449616093056/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572345 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572335.694044:0:2378:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572335.694047:0:2378:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada456be300 x1875449616093056/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572345 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572335.694055:0:2378:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572335.694058:0:2378:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000001:2.0:1788572335.696020:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572335.696044:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788572335.696051:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572335.696053:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572335.696063:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572335.696073:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:2.0:1788572335.696085:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572335.696090:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572335.696096:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572335.696102:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 416 into portal 4 MB=0x6a9b624294f80 00000400:00000200:2.0:1788572335.696108:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572335.696115:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.202.113@tcp of length 416/416 into md 0x357d5 [1] + 0 00000800:00000001:2.0:1788572335.696124:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572335.696133:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572335.696137:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572335.696142:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:2.0:1788572335.696153:0:2377:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:2.0:1788572335.696162:0:2377:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada456be300 x1875449616093056/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572345 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788572335.696177:0:2377:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=416 offset=0 replen=544 req@ffff9ada456be300 x1875449616093056/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572345 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572335.696205:0:2377:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788572335.696215:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572335.696219:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572335.696297:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788572335.696302:0:2382:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788572335.696309:0:2382:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788572335.696311:0:2382:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788572335.696315:0:2382:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada43d96a50. 00000400:00000200:0.0:1788572335.696320:0:2382:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada43c8faa0 00000400:00000010:0.0:1788572335.696323:0:2382:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada43c8faa0. 00000100:00000001:0.0:1788572335.696326:0:2382:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:0.0:1788572335.696331:0:2382:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada456be300 x1875449616093056/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572345 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788572335.696352:0:2382:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada456be300 x1875449616093056/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 1788572345 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572335.696358:0:2382:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788572335.696360:0:2382:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572335.696363:0:2382:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:0.0:1788572335.696366:0:2382:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788572335.696369:0:2382:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788572335.696372:0:2382:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572335.696375:0:2382:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788572335.696387:0:2382:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada48407c60 time=180 v=5 (5 5 5 5) 00000100:00001000:0.0:1788572335.696755:0:2382:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada48407c18 time=57 v=5 (20 1 1 1) 00000100:00000001:0.0:1788572335.696758:0:2382:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788572335.696761:0:2382:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788572335.696764:0:2382:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:0.0:1788572335.696770:0:2382:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff9ada456be300 x1875449616093056/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/416 e 0 to 0 dl 1788572345 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788572335.696777:0:2382:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572335.696781:0:2382:0:(client.c:3110:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788572335.696784:0:2382:0:(client.c:3122:ptlrpc_free_committed()) lustre-OST0000-osc-ffff9ada58b0a000: skip recheck: last_committed 8589935763 00000100:00000001:0.0:1788572335.696786:0:2382:0:(client.c:3123:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788572335.696788:0:2382:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572335.696793:0:2382:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada456be300 x1875449616093056/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/416 e 0 to 0 dl 1788572345 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572335.696800:0:2382:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788572335.696803:0:2382:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788572335.696805:0:2382:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572335.696808:0:2382:0:(import.c:1101:ptlrpc_connect_interpret()) Process entered 00000100:00080000:0.0:1788572335.696816:0:2382:0:(import.c:1176:ptlrpc_connect_interpret()) lustre-OST0000-osc-ffff9ada58b0a000: connect to target with instance 3 00000020:00000001:0.0:1788572335.696819:0:2382:0:(genops.c:792:class_conn2export()) Process entered 00000020:00000040:0.0:1788572335.696832:0:2382:0:(genops.c:804:class_conn2export()) looking for export cookie 0xef30c340bfb39c83 00000020:00000001:0.0:1788572335.696835:0:2382:0:(lustre_handles.c:130:class_handle2object()) Process entered 00000020:00000040:0.0:1788572335.696839:0:2382:0:(lustre_handles.c:148:class_handle2object()) GET export ffff9ada50c3b800 refcount=4 00000020:00000001:0.0:1788572335.696841:0:2382:0:(lustre_handles.c:155:class_handle2object()) Process leaving (rc=18446632861181392896 : -111212528158720 : ffff9ada50c3b800) 00000020:00000001:0.0:1788572335.696844:0:2382:0:(genops.c:806:class_conn2export()) Process leaving (rc=18446632861181392896 : -111212528158720 : ffff9ada50c3b800) 00000100:00001000:0.0:1788572335.696847:0:2382:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada48407c18 time=1788572335 v=0 (0 0 0 0) 00000100:00001000:0.0:1788572335.696850:0:2382:0:(import.c:2261:obd_at_measure()) AT ffff9ada48407c18 change: old=0 new=5 delta=5 (val=1) hist 1 0 0 0 00000100:00000040:0.0:1788572335.696900:0:2382:0:(obd_cksum.h:77:obd_cksum_types_supported_client()) Crypto hash speed: crc 1623, crc32c 10419, adler 1295 00000020:00000040:0.0:1788572335.696907:0:2382:0:(genops.c:879:class_export_put()) PUTting export ffff9ada50c3b800 : new refcount 3 00000100:00000001:0.0:1788572335.696909:0:2382:0:(obd_class.h:1222:obd_import_event()) Process entered 00000008:00000001:0.0:1788572335.696913:0:2382:0:(osc_request.c:3902:osc_import_event()) Process entered 00000008:00000020:0.0:1788572335.696917:0:2382:0:(osc_request.c:1062:osc_init_grant()) lustre-OST0000-osc-ffff9ada58b0a000, setting cl_avail_grant: 8388608 cl_lost_grant: 0. chunk bits: 12 cl_max_extent_pages: 16384 00000008:00000001:0.0:1788572335.696921:0:2382:0:(obd_class.h:1242:obd_notify()) Process entered 00020000:00000001:0.0:1788572335.696924:0:2382:0:(lov_obd.c:430:lov_notify()) Process entered 00000080:00000001:0.0:1788572335.696929:0:2382:0:(llite_lib.c:386:ll_upcall_notify()) Process entered 00000080:00000001:0.0:1788572335.696932:0:2382:0:(llite_lib.c:411:ll_upcall_notify()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:0.0:1788572335.696935:0:2382:0:(lov_obd.c:467:lov_notify()) Process leaving (rc=0 : 0 : 0) 00000008:00000001:0.0:1788572335.696936:0:2382:0:(obd_class.h:1255:obd_notify()) Process leaving (rc=0 : 0 : 0) 00000008:00000001:0.0:1788572335.696939:0:2382:0:(osc_request.c:3969:osc_import_event()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572335.696940:0:2382:0:(obd_class.h:1233:obd_import_event()) Process leaving 00000100:00080000:0.0:1788572335.696943:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada48407800 lustre-OST0000_UUID: changing import state from CONNECTING to REPLAY 00000100:00000001:0.0:1788572335.696948:0:2382:0:(import.c:1692:ptlrpc_import_recovery_state_machine()) Process entered 00000100:00080000:0.0:1788572335.696951:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-OST0000_UUID 00000100:00000001:0.0:1788572335.696953:0:2382:0:(recover.c:40:ptlrpc_replay_next()) Process entered 00000100:00000001:0.0:1788572335.696954:0:2382:0:(client.c:3110:ptlrpc_free_committed()) Process entered 00000100:00100000:0.0:1788572335.696956:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-ffff9ada58b0a000: committing for last_committed 8589935763 gen 5 00000100:00000040:0.0:1788572335.696959:0:2382:0:(client.c:3236:ptlrpc_free_committed()) lustre-OST0000-osc-ffff9ada58b0a000: normal: skip=0 replay=0/0 committed=0/0 00000100:00000001:0.0:1788572335.696962:0:2382:0:(client.c:3237:ptlrpc_free_committed()) Process leaving 00000100:00080000:0.0:1788572335.696963:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada48407800 from lustre-OST0000_UUID committed 8589935763 last 0 00000100:00000001:0.0:1788572335.696966:0:2382:0:(recover.c:138:ptlrpc_replay_next()) Process leaving (rc=0 : 0 : 0) 00000100:00080000:0.0:1788572335.696968:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada48407800 lustre-OST0000_UUID: changing import state from REPLAY to REPLAY_LOCKS 00000020:00000040:0.0:1788572335.696971:0:2382:0:(genops.c:1210:class_import_get()) import ffff9ada48407800 refcount=4 obd=lustre-OST0000-osc-ffff9ada58b0a000 00000100:00000001:0.0:1788572335.697781:0:2382:0:(import.c:1798:ptlrpc_import_recovery_state_machine()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572335.697790:0:2382:0:(import.c:1561:ptlrpc_connect_interpret()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572335.697800:0:2382:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada456be300 x1875449616093056/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/416 e 0 to 0 dl 1788572345 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572335.697809:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada456be300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616093056:192.168.202.113@tcp:8:ll_cfg_requeue.0 00000100:00000001:0.0:1788572335.697814:0:2382:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572335.697818:0:2382:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788572335.697832:0:2382:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada456be300 x1875449616093056/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/416 e 0 to 0 dl 1788572345 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'ll_cfg_requeue.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572335.697839:0:2382:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788572335.697843:0:2382:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788572335.697848:0:2382:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 1024 at ffff9ada6056ac00. 02000000:00000001:0.0:1788572335.697877:0:2382:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788572335.697880:0:2382:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:0.0:1788572335.697882:0:2382:0:(genops.c:1223:class_import_put()) import ffff9ada48407800 refcount=3 obd=lustre-OST0000-osc-ffff9ada58b0a000 00000020:00000001:0.0:1788572335.697884:0:2382:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:0.0:1788572335.697887:0:2382:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 1024 at ffff9ada454bb400. 02000000:00000001:0.0:1788572335.697891:0:2382:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788572335.697893:0:2382:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788572335.697895:0:2382:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada456be300. 00000100:00000001:0.0:1788572335.697899:0:2382:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788572335.697902:0:2382:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572335.697904:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572335.697914:0:2382:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788572335.697916:0:2382:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572335.697918:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788572335.697920:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00080000:2.0:1788572335.697969:0:88948:0:(ldlm_request.c:2972:ldlm_lock_replay_thread()) lock replay thread lustre-OST0000-osc-ffff9ada58b0a000 to lustre-OST0000_UUID@192.168.202.113@tcp 00010000:00000001:2.0:1788572335.697973:0:88948:0:(ldlm_request.c:2924:__ldlm_replay_locks()) Process entered 00010000:00010000:2.0:1788572335.697975:0:88948:0:(ldlm_request.c:2888:ldlm_cancel_unused_locks_for_replay()) Dropping as many unused locks as possible before replay for namespace lustre-OST0000-osc-ffff9ada58b0a000 (1) 00010000:00000001:2.0:1788572335.697981:0:88948:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00000008:00000001:2.0:1788572335.697990:0:88948:0:(osc_lock.c:730:osc_ldlm_weigh_ast()) Process entered 00000020:00000001:2.0:1788572335.697993:0:88948:0:(cl_object.c:754:cl_env_obtain()) Process entered 00000020:00000001:2.0:1788572335.697997:0:88948:0:(cl_object.c:785:cl_env_obtain()) Process leaving (rc=18446632861448591192 : -111212260960424 : ffff9ada60b0d758) 00000020:00001000:2.0:1788572335.698000:0:88948:0:(cl_object.c:823:cl_env_get()) 1@ffff9ada60b0d750 00000020:00000001:2.0:1788572335.698006:0:88948:0:(cl_object.c:143:cl_object_top()) ffff9ada60acd340 -> ffff9ada607dfdc8 00000008:00000001:2.0:1788572335.698008:0:88948:0:(osc_lock.c:694:osc_lock_weight()) Process entered 00000020:00000001:2.0:1788572335.698010:0:88948:0:(cl_object.c:143:cl_object_top()) ffff9ada607dfdc8 -> ffff9ada607dfdc8 00000020:00000001:2.0:1788572335.698012:0:88948:0:(cl_io.c:125:__cl_io_init()) Process entered 00000080:00000001:2.0:1788572335.698015:0:88948:0:(vvp_io.c:1821:vvp_io_init()) Process entered 00000080:00200000:2.0:1788572335.698017:0:88948:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x643:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000020:00000001:2.0:1788572335.698020:0:88948:0:(cl_io.c:898:cl_io_slice_add()) Process entered 00000020:00000001:2.0:1788572335.698022:0:88948:0:(cl_io.c:904:cl_io_slice_add()) Process leaving 00000080:00000001:2.0:1788572335.698024:0:88948:0:(vvp_io.c:1898:vvp_io_init()) Process leaving (rc=0 : 0 : 0) 00020000:00000002:2.0:1788572335.698029:0:88948:0:(lov_object.c:1590:lov_io_init()) [0x2000182d1:0x643:0x0]io ffff9ada60791700 type 8 ignore/verify layout 1/0 00020000:00000001:2.0:1788572335.698033:0:88948:0:(lov_io.c:2181:lov_io_init_composite()) Process entered 00020000:00000001:2.0:1788572335.698035:0:88948:0:(lov_io.c:721:lov_io_slice_init()) Process entered 00020000:00000001:2.0:1788572335.698038:0:88948:0:(lov_io.c:825:lov_io_slice_init()) Process leaving via out (rc=0 : 0 : 0x0) 00020000:00000001:2.0:1788572335.698040:0:88948:0:(lov_io.c:206:lov_io_subio_init()) Process entered 00020000:00000001:2.0:1788572335.698042:0:88948:0:(lov_io.c:214:lov_io_subio_init()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788572335.698044:0:88948:0:(cl_io.c:898:cl_io_slice_add()) Process entered 00000020:00000001:2.0:1788572335.698045:0:88948:0:(cl_io.c:904:cl_io_slice_add()) Process leaving 00020000:00000001:2.0:1788572335.698046:0:88948:0:(lov_io.c:2194:lov_io_init_composite()) Process leaving 00000020:00000001:2.0:1788572335.698047:0:88948:0:(cl_io.c:142:__cl_io_init()) Process leaving (rc=0 : 0 : 0) 00000008:00000001:2.0:1788572335.698050:0:88948:0:(osc_cache.c:3516:osc_page_gang_lookup()) Process entered 00000020:00000001:2.0:1788572335.698055:0:88948:0:(cl_page.c:534:cl_page_get()) Process entered 00000020:00000001:2.0:1788572335.698056:0:88948:0:(cl_page.c:536:cl_page_get()) Process leaving 00000020:00000001:2.0:1788572335.698058:0:88948:0:(cl_page.c:554:cl_batch_put()) Process entered 00000020:00000001:2.0:1788572335.698065:0:88948:0:(cl_page.c:556:cl_batch_put()) page@000000000bc7d1a7[2 000000001b009600 1 1 0000000000000000 0 0 0] 00000020:00000001:2.0:1788572335.698067:0:88948:0:(cl_page.c:556:cl_batch_put()) 2 00000020:00000001:2.0:1788572335.698068:0:88948:0:(cl_page.c:569:cl_batch_put()) Process leaving 00000008:00000001:2.0:1788572335.698070:0:88948:0:(osc_cache.c:3593:osc_page_gang_lookup()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:2.0:1788572335.698072:0:88948:0:(cl_io.c:71:cl_io_fini()) Process entered 00020000:00000001:2.0:1788572335.698075:0:88948:0:(lov_io.c:900:lov_io_fini()) Process entered 00020000:00000001:2.0:1788572335.698077:0:88948:0:(lov_io.c:919:lov_io_fini()) Process leaving 00000080:00000001:2.0:1788572335.698079:0:88948:0:(vvp_io.c:296:vvp_io_fini()) Process entered 00000080:00200000:2.0:1788572335.698082:0:88948:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x643:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00000001:2.0:1788572335.698085:0:88948:0:(vvp_io.c:402:vvp_io_fini()) Process leaving 00000020:00000001:2.0:1788572335.698087:0:88948:0:(cl_io.c:112:cl_io_fini()) Process leaving 00000008:00000001:2.0:1788572335.698089:0:88948:0:(osc_lock.c:784:osc_ldlm_weigh_ast()) Process leaving 00000020:00001000:2.0:1788572335.698092:0:88948:0:(cl_object.c:917:cl_env_put()) 1@ffff9ada60b0d750 00000008:00000001:2.0:1788572335.698095:0:88948:0:(osc_request.c:3991:osc_cancel_weight()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:2.0:1788572335.698098:0:88948:0:(ldlm_request.c:1930:ldlm_cancel_no_wait_policy()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572335.698101:0:88948:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:2.0:1788572335.698104:0:88948:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:2.0:1788572335.698106:0:88948:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:2.0:1788572335.698109:0:88948:0:(ldlm_lock.c:2501:ldlm_lock_cancel()) Process entered 00000008:00000001:2.0:1788572335.698113:0:88948:0:(osc_lock.c:529:osc_ldlm_blocking_ast()) Process entered 00000020:00000001:2.0:1788572335.698114:0:88948:0:(cl_object.c:754:cl_env_obtain()) Process entered 00000020:00000001:2.0:1788572335.698116:0:88948:0:(cl_object.c:785:cl_env_obtain()) Process leaving (rc=18446632861448591192 : -111212260960424 : ffff9ada60b0d758) 00000020:00001000:2.0:1788572335.698118:0:88948:0:(cl_object.c:823:cl_env_get()) 1@ffff9ada60b0d750 00000008:00000001:2.0:1788572335.698120:0:88948:0:(osc_lock.c:413:__osc_dlm_blocking_ast()) Process entered 00000008:00000001:2.0:1788572335.698123:0:88948:0:(osc_lock.c:355:osc_lock_flush()) Process entered 00000020:00000001:2.0:1788572335.698124:0:88948:0:(cl_object.c:754:cl_env_obtain()) Process entered 00000020:00000001:2.0:1788572335.698126:0:88948:0:(cl_object.c:785:cl_env_obtain()) Process leaving (rc=18446632861448593064 : -111212260958552 : ffff9ada60b0dea8) 00000020:00001000:2.0:1788572335.698128:0:88948:0:(cl_object.c:823:cl_env_get()) 1@ffff9ada60b0dea0 00000008:00000001:2.0:1788572335.698130:0:88948:0:(osc_cache.c:3760:osc_ldlm_hp_handle()) Process entered 00000008:00000020:2.0:1788572335.698132:0:88948:0:(osc_cache.c:3780:osc_ldlm_hp_handle()) High prio I/O check: start 0 end 4503599627370495 RPC(0):r0/w0/d0 00000008:00000001:2.0:1788572335.698136:0:88948:0:(osc_cache.c:3796:osc_ldlm_hp_handle()) Process leaving via out_unlock (rc=0 : 0 : 0x0) 00000008:00000001:2.0:1788572335.698138:0:88948:0:(osc_cache.c:3844:osc_ldlm_hp_handle()) Process leaving (rc=0 : 0 : 0) 00000008:00000001:2.0:1788572335.698141:0:88948:0:(osc_cache.c:3729:osc_lock_discard_pages()) Process entered 00000020:00000001:2.0:1788572335.698142:0:88948:0:(cl_object.c:143:cl_object_top()) ffff9ada60acd340 -> ffff9ada607dfdc8 00000020:00000001:2.0:1788572335.698144:0:88948:0:(cl_object.c:143:cl_object_top()) ffff9ada607dfdc8 -> ffff9ada607dfdc8 00000020:00000001:2.0:1788572335.698146:0:88948:0:(cl_io.c:125:__cl_io_init()) Process entered 00000080:00000001:2.0:1788572335.698147:0:88948:0:(vvp_io.c:1821:vvp_io_init()) Process entered 00000080:00200000:2.0:1788572335.698149:0:88948:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x643:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000020:00000001:2.0:1788572335.698151:0:88948:0:(cl_io.c:898:cl_io_slice_add()) Process entered 00000020:00000001:2.0:1788572335.698152:0:88948:0:(cl_io.c:904:cl_io_slice_add()) Process leaving 00000080:00000001:2.0:1788572335.698154:0:88948:0:(vvp_io.c:1898:vvp_io_init()) Process leaving (rc=0 : 0 : 0) 00020000:00000002:2.0:1788572335.698156:0:88948:0:(lov_object.c:1590:lov_io_init()) [0x2000182d1:0x643:0x0]io ffff9ada60792230 type 8 ignore/verify layout 1/0 00020000:00000001:2.0:1788572335.698159:0:88948:0:(lov_io.c:2181:lov_io_init_composite()) Process entered 00020000:00000001:2.0:1788572335.698160:0:88948:0:(lov_io.c:721:lov_io_slice_init()) Process entered 00020000:00000001:2.0:1788572335.698162:0:88948:0:(lov_io.c:825:lov_io_slice_init()) Process leaving via out (rc=0 : 0 : 0x0) 00020000:00000001:2.0:1788572335.698164:0:88948:0:(lov_io.c:206:lov_io_subio_init()) Process entered 00020000:00000001:2.0:1788572335.698165:0:88948:0:(lov_io.c:214:lov_io_subio_init()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788572335.698166:0:88948:0:(cl_io.c:898:cl_io_slice_add()) Process entered 00000020:00000001:2.0:1788572335.698168:0:88948:0:(cl_io.c:904:cl_io_slice_add()) Process leaving 00020000:00000001:2.0:1788572335.698169:0:88948:0:(lov_io.c:2194:lov_io_init_composite()) Process leaving 00000020:00000001:2.0:1788572335.698170:0:88948:0:(cl_io.c:142:__cl_io_init()) Process leaving (rc=0 : 0 : 0) 00000008:00000001:2.0:1788572335.698172:0:88948:0:(osc_cache.c:3516:osc_page_gang_lookup()) Process entered 00000020:00000001:2.0:1788572335.698174:0:88948:0:(cl_page.c:534:cl_page_get()) Process entered 00000020:00000001:2.0:1788572335.698175:0:88948:0:(cl_page.c:536:cl_page_get()) Process leaving 00000008:00000001:2.0:1788572335.698178:0:88948:0:(osc_lock.c:1343:osc_obj_dlmlock_at_pgoff()) Process entered 00000008:00000001:2.0:1788572335.698182:0:88948:0:(osc_request.c:3331:osc_match_base()) Process entered 00010000:00000001:2.0:1788572335.698185:0:88948:0:(ldlm_lock.c:1446:ldlm_lock_match_with_skip()) Process entered 00010000:00000040:2.0:1788572335.698192:0:88948:0:(ldlm_resource.c:1656:ldlm_resource_getref()) getref res: ffff9ada60762d00 count: 2 00010000:00000001:2.0:1788572335.698195:0:88948:0:(ldlm_resource.c:1595:ldlm_resource_get()) Process leaving via found (rc=0 : 0 : 0x0) 00000008:00000001:2.0:1788572335.698203:0:88948:0:(osc_request.c:3346:osc_match_base()) Process leaving (rc=0 : 0 : 0) 00000008:00000001:2.0:1788572335.698205:0:88948:0:(osc_lock.c:1377:osc_obj_dlmlock_at_pgoff()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788572335.698208:0:88948:0:(cl_page.c:700:__cl_page_own()) Process entered 00000020:00000001:2.0:1788572335.698212:0:88948:0:(cl_io.c:1145:cl_io_top()) Process entered 00000020:00000001:2.0:1788572335.698213:0:88948:0:(cl_io.c:1148:cl_io_top()) Process leaving (rc=18446632861444940336 : -111212264611280 : ffff9ada60792230) 00000020:00000001:2.0:1788572335.698215:0:88948:0:(cl_page.c:634:cl_page_owner_set()) Process entered 00000020:00000001:2.0:1788572335.698217:0:88948:0:(cl_page.c:637:cl_page_owner_set()) Process leaving 00000020:00000001:2.0:1788572335.698219:0:88948:0:(cl_page.c:504:__cl_page_state_set()) Process entered 00000020:00000001:2.0:1788572335.698223:0:88948:0:(cl_page.c:507:__cl_page_state_set()) page@000000000bc7d1a7[2 000000001b009600 1 1 0000000049fcfbf6 0 0 0] 00000020:00000001:2.0:1788572335.698225:0:88948:0:(cl_page.c:507:__cl_page_state_set()) 1 -> 2 00000020:00000001:2.0:1788572335.698227:0:88948:0:(cl_page.c:515:__cl_page_state_set()) Process leaving 00000020:00000040:2.0:1788572335.698229:0:88948:0:(cl_page.c:744:__cl_page_own()) res 0 00000020:00000001:2.0:1788572335.698230:0:88948:0:(cl_page.c:747:__cl_page_own()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788572335.698233:0:88948:0:(cl_page.c:976:cl_page_delete()) Process entered 00000020:00000001:2.0:1788572335.698234:0:88948:0:(cl_page.c:928:__cl_page_delete()) Process entered 00000020:00000001:2.0:1788572335.698236:0:88948:0:(cl_page.c:623:cl_page_owner_clear()) Process entered 00000020:00000001:2.0:1788572335.698237:0:88948:0:(cl_page.c:629:cl_page_owner_clear()) Process leaving 00000020:00000001:2.0:1788572335.698238:0:88948:0:(cl_page.c:504:__cl_page_state_set()) Process entered 00000020:00000001:2.0:1788572335.698241:0:88948:0:(cl_page.c:507:__cl_page_state_set()) page@000000000bc7d1a7[2 000000001b009600 2 1 0000000000000000 0 0 0] 00000020:00000001:2.0:1788572335.698243:0:88948:0:(cl_page.c:507:__cl_page_state_set()) 2 -> 5 00000020:00000001:2.0:1788572335.698244:0:88948:0:(cl_page.c:515:__cl_page_state_set()) Process leaving 00000008:00000001:2.0:1788572335.698246:0:88948:0:(osc_page.c:137:osc_page_delete()) Process entered 00000008:00000001:2.0:1788572335.698247:0:88948:0:(osc_page.c:138:osc_page_delete()) ffff9ada42f34d80 00000008:00000001:2.0:1788572335.698250:0:88948:0:(osc_cache.c:2730:osc_teardown_async_page()) Process entered 00000008:00000040:2.0:1788572335.698251:0:88948:0:(osc_cache.c:2733:osc_teardown_async_page()) teardown oap ffff9ada42f34d90 page ffff9ada42f34d80 at index 0. 00000008:00000001:2.0:1788572335.698255:0:88948:0:(osc_cache.c:2755:osc_teardown_async_page()) Process leaving (rc=0 : 0 : 0) 00000008:00000001:2.0:1788572335.698263:0:88948:0:(osc_page.c:169:osc_page_delete()) Process leaving 00000080:00000020:2.0:1788572335.698266:0:88948:0:(vvp_page.c:56:vvp_page_delete()) delete page ffffe340c4997b40 index 0 00000020:00000001:2.0:1788572335.698272:0:88948:0:(cl_page.c:946:__cl_page_delete()) Process leaving 00000020:00000001:2.0:1788572335.698274:0:88948:0:(cl_page.c:978:cl_page_delete()) Process leaving 00000020:00000001:2.0:1788572335.698286:0:88948:0:(cl_page.c:645:__cl_page_disown()) Process entered 00000020:00000001:2.0:1788572335.698288:0:88948:0:(cl_page.c:623:cl_page_owner_clear()) Process entered 00000020:00000001:2.0:1788572335.698289:0:88948:0:(cl_page.c:629:cl_page_owner_clear()) Process leaving 00000020:00000001:2.0:1788572335.698291:0:88948:0:(cl_page.c:660:__cl_page_disown()) Process leaving 00000020:00000001:2.0:1788572335.698292:0:88948:0:(cl_page.c:554:cl_batch_put()) Process entered 00000020:00000001:2.0:1788572335.698295:0:88948:0:(cl_page.c:556:cl_batch_put()) page@000000000bc7d1a7[1 000000001b009600 5 1 0000000000000000 0 0 0] 00000020:00000001:2.0:1788572335.698296:0:88948:0:(cl_page.c:556:cl_batch_put()) 1 00000020:00000001:2.0:1788572335.698298:0:88948:0:(cl_page.c:236:cl_page_free()) Process entered 00000020:00000010:2.0:1788572335.698303:0:88948:0:(cl_page.c:223:__cl_page_free()) slab-freed 'cl_page': 224 at ffff9ada42f34d20. 00000020:00000001:2.0:1788572335.698307:0:88948:0:(cl_page.c:274:cl_page_free()) Process leaving 00000020:00000001:2.0:1788572335.698308:0:88948:0:(cl_page.c:569:cl_batch_put()) Process leaving 00000008:00000001:2.0:1788572335.700791:0:88948:0:(osc_cache.c:3593:osc_page_gang_lookup()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:2.0:1788572335.700804:0:88948:0:(cl_io.c:71:cl_io_fini()) Process entered 00020000:00000001:2.0:1788572335.700811:0:88948:0:(lov_io.c:900:lov_io_fini()) Process entered 00020000:00000001:2.0:1788572335.700814:0:88948:0:(lov_io.c:919:lov_io_fini()) Process leaving 00000080:00000001:2.0:1788572335.700818:0:88948:0:(vvp_io.c:296:vvp_io_fini()) Process entered 00000080:00200000:2.0:1788572335.700831:0:88948:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x643:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 1 00000080:00000001:2.0:1788572335.700834:0:88948:0:(vvp_io.c:402:vvp_io_fini()) Process leaving 00000020:00000001:2.0:1788572335.700835:0:88948:0:(cl_io.c:112:cl_io_fini()) Process leaving 00000008:00000001:2.0:1788572335.700837:0:88948:0:(osc_cache.c:3748:osc_lock_discard_pages()) Process leaving (rc=0 : 0 : 0) 00000020:00001000:2.0:1788572335.700842:0:88948:0:(cl_object.c:917:cl_env_put()) 1@ffff9ada60b0dea0 00000008:00000001:2.0:1788572335.700848:0:88948:0:(osc_lock.c:387:osc_lock_flush()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788572335.700878:0:88948:0:(cl_object.c:143:cl_object_top()) ffff9ada60acd340 -> ffff9ada607dfdc8 00010000:00000001:2.0:1788572335.700882:0:88948:0:(ldlm_extent.c:945:ldlm_extent_shift_kms()) Process entered 00010000:00000001:2.0:1788572335.700885:0:88948:0:(ldlm_extent.c:1018:ldlm_extent_shift_kms()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788572335.700887:0:88948:0:(cl_object.c:143:cl_object_top()) ffff9ada60acd340 -> ffff9ada607dfdc8 00000020:00000001:2.0:1788572335.700889:0:88948:0:(cl_object.c:248:cl_object_attr_update()) Process entered 00000008:00000020:2.0:1788572335.700892:0:88948:0:(osc_object.c:161:osc_attr_update()) set kms from 4096to 0 00020000:00000001:2.0:1788572335.700895:0:88948:0:(lovsub_object.c:98:lovsub_attr_update()) Process entered 00020000:00000001:2.0:1788572335.700898:0:88948:0:(lovsub_object.c:100:lovsub_attr_update()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788572335.700899:0:88948:0:(cl_object.c:261:cl_object_attr_update()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788572335.700901:0:88948:0:(cl_object.c:143:cl_object_top()) ffff9ada60acd340 -> ffff9ada607dfdc8 00000020:00000001:2.0:1788572335.700903:0:88948:0:(cl_object.c:143:cl_object_top()) ffff9ada607dfdc8 -> ffff9ada607dfdc8 00000020:00000001:2.0:1788572335.700905:0:88948:0:(cl_object.c:248:cl_object_attr_update()) Process entered 00000020:00000001:2.0:1788572335.700908:0:88948:0:(cl_object.c:261:cl_object_attr_update()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788572335.700909:0:88948:0:(cl_object.c:143:cl_object_top()) ffff9ada60acd340 -> ffff9ada607dfdc8 00000008:00000001:2.0:1788572335.700914:0:88948:0:(osc_lock.c:470:__osc_dlm_blocking_ast()) Process leaving (rc=0 : 0 : 0) 00000020:00001000:2.0:1788572335.700917:0:88948:0:(cl_object.c:917:cl_env_put()) 1@ffff9ada60b0d750 00000008:00000001:2.0:1788572335.700920:0:88948:0:(osc_lock.c:569:osc_ldlm_blocking_ast()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572335.700927:0:88948:0:(ldlm_lock.c:410:ldlm_lock_destroy_nolock()) Process entered 00010000:00000001:2.0:1788572335.700929:0:88948:0:(ldlm_lock.c:354:ldlm_lock_destroy_internal()) Process entered 00010000:00000001:2.0:1788572335.700932:0:88948:0:(ldlm_lock.c:261:ldlm_lock_remove_from_lru_check()) Process entered 00010000:00000001:2.0:1788572335.700935:0:88948:0:(ldlm_lock.c:280:ldlm_lock_remove_from_lru_check()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:2.0:1788572335.700940:0:88948:0:(lustre_handles.c:107:class_handle_unhash_nolock()) removing object ffff9ada6035ca00 with handle 0xef30c340bfb47089 from hash 00010000:00000001:2.0:1788572335.700942:0:88948:0:(ldlm_lock.c:384:ldlm_lock_destroy_internal()) Process leaving 00010000:00000001:2.0:1788572335.700944:0:88948:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:2.0:1788572335.700946:0:88948:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:2.0:1788572335.700947:0:88948:0:(ldlm_lock.c:416:ldlm_lock_destroy_nolock()) Process leaving 00010000:00000001:2.0:1788572335.700953:0:88948:0:(ldlm_lock.c:2537:ldlm_lock_cancel()) Process leaving 00010000:00000001:2.0:1788572335.700955:0:88948:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00010000:2.0:1788572335.700959:0:88948:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff9ada58b0a000 lock: 00000000bf5c7bf0/0xef30c340bfb47089 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x9a2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x804869400000000 nid: local remote: 0xdb7edb52b83267bb expref: -99 pid: 87653 timeout: 0 lvb_type: 1 lru_score: 1 lru_type: 0 00010000:00000010:2.0:1788572335.700973:0:88948:0:(ldlm_lock.c:209:ldlm_lock_put()) kvfree 'lock->l_lvb_data': 56 at ffff9ada60aacf80. 00010000:00000040:2.0:1788572335.700980:0:88948:0:(ldlm_resource.c:1704:ldlm_resource_putref()) putref res: ffff9ada60762d00 count: 0 00010000:00000010:2.0:1788572335.700988:0:88948:0:(ldlm_resource.c:1563:ldlm_resource_free()) slab-freed 'res->lr_itree': 216 at ffff9ada60762100. 00010000:00000001:2.0:1788572335.700998:0:88948:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:2.0:1788572335.701000:0:88948:0:(ldlm_request.c:1890:ldlm_cli_cancel_list_local()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788572335.701003:0:88948:0:(ldlm_request.c:2901:ldlm_cancel_unused_locks_for_replay()) Canceled 0 unused locks from namespace lustre-OST0000-osc-ffff9ada58b0a000 00000001:00000001:2.0:1788572335.701007:0:88948:0:(hash.c:1696:cfs_hash_for_each_nolock()) Process entered 00000001:00000001:2.0:1788572335.701010:0:88948:0:(hash.c:1609:cfs_hash_for_each_relax()) Process entered 00000001:00000001:2.0:1788572335.701046:0:88948:0:(hash.c:1712:cfs_hash_for_each_nolock()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572335.701048:0:88948:0:(ldlm_request.c:2954:__ldlm_replay_locks()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572335.701051:0:88948:0:(import.c:1692:ptlrpc_import_recovery_state_machine()) Process entered 00000100:00080000:2.0:1788572335.701055:0:88948:0:(import.c:65:import_set_state_nolock()) ffff9ada48407800 lustre-OST0000_UUID: changing import state from REPLAY_LOCKS to REPLAY_WAIT 00000100:00000001:2.0:1788572335.701058:0:88948:0:(import.c:1601:signal_completed_replay()) Process entered 00000100:00000010:3.0:1788572335.704249:0:88948:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada455a4700. 00000020:00000040:3.0:1788572335.704731:0:88948:0:(genops.c:1210:class_import_get()) import ffff9ada48407800 refcount=4 obd=lustre-OST0000-osc-ffff9ada58b0a000 00000100:00000001:3.0:1788572335.704739:0:88948:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788572335.704742:0:88948:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788572335.704749:0:88948:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788572335.704761:0:88948:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada50c31400. 00000100:00000001:3.0:1788572335.704770:0:88948:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572335.704776:0:88948:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788572335.704782:0:88948:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572335.704795:0:88948:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000e95cad70] to pc [ptlrpcd_rcv+-1] req@ffff9ada455a4700 x1875449616093184/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:QU/2c0/ffffffff rc 0/-1 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572335.704835:0:88948:0:(import.c:1626:signal_completed_replay()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572335.704839:0:88948:0:(import.c:1798:ptlrpc_import_recovery_state_machine()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788572335.704842:0:88948:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:3.0:1788572335.704843:0:88948:0:(genops.c:1223:class_import_put()) import ffff9ada48407800 refcount=3 obd=lustre-OST0000-osc-ffff9ada58b0a000 00000020:00000001:3.0:1788572335.704846:0:88948:0:(genops.c:1230:class_import_put()) Process leaving 00000100:00000001:0.0:1788572335.705258:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788572335.705264:0:2382:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788572335.705267:0:2382:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788572335.705274:0:2382:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada455a4700 x1875449616093184/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:QU/2c0/ffffffff rc 0/-1 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572335.705284:0:2382:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788572335.705287:0:2382:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572335.705291:0:2382:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788572335.705294:0:2382:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788572335.705297:0:2382:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788572335.705301:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616093184:192.168.202.113@tcp:400:ldlm_lock_repla.0 00000100:00000001:0.0:1788572335.705306:0:2382:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788572335.705312:0:2382:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788572335.705315:0:2382:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572335.705318:0:2382:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788572335.705324:0:2382:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada456bb800. 02000000:00000001:0.0:1788572335.705327:0:2382:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788572335.705335:0:2382:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:0.0:1788572335.705341:0:2382:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada60acaf20. 00000400:00000010:0.0:1788572335.705350:0:2382:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada5176aaa0. 00000100:00000200:0.0:1788572335.705358:0:2382:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616093184, portal 4 00000100:00000001:0.0:1788572335.705361:0:2382:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788572335.705362:0:2382:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860989933312 : -111212719618304 : ffff9ada455a4700) 00000100:00000040:0.0:1788572335.705371:0:2382:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=2c0 req@ffff9ada455a4700 x1875449616093184/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572351 ref 2 fl Rpc:r/2c0/ffffffff rc 0/-1 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572335.705378:0:2382:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788572335.705382:0:2382:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:0.0:1788572335.705384:0:2382:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada5176a770. 00000100:00000200:0.0:1788572335.705389:0:2382:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1875449616093184, offset 0 00000400:00000200:0.0:1788572335.705398:0:2382:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:0.0:1788572335.705407:0:2382:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:0.0:1788572335.705417:0:2382:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:0.0:1788572335.705424:0:2382:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788572335.705429:0:2382:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:0.0:1788572335.705433:0:2382:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788572335.705436:0:2382:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:0.0:1788572335.705451:0:2382:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22300:22300:256:4294967295] 192.168.202.113@tcp LPNI seq info [22296:22296:8:4294967295] 00000400:00000200:0.0:1788572335.705456:0:2382:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:0.0:1788572335.705467:0:2382:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:0.0:1788572335.705473:0:2382:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:0.0:1788572335.705479:0:2382:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada5074a000. 00000800:00000200:0.0:1788572335.705486:0:2382:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:0.0:1788572335.705495:0:2382:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:0.0:1788572335.705500:0:2382:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada5074a000 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788572335.705520:0:2382:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572335.705526:0:2382:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: using a timeout of 18878 usecs (19 jiffies) 00000400:00000200:0.0:1788572335.705529:0:2382:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:0.0:1788572335.705533:0:2382:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298920693 00000100:00000001:0.0:1788572335.705536:0:2382:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298931216 00000100:00000001:0.0:1788572335.705539:0:2382:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298920693 00000100:00000001:0.0:1788572335.705542:0:2382:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298931216 00000100:00000001:0.0:1788572335.705544:0:2382:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788572335.705547:0:2382:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572335.705551:0:2382:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572335.705554:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572335.705571:0:2382:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788572335.705573:0:2382:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788572335.705575:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788572335.705577:0:2382:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788572335.705578:0:2382:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572335.705579:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788572335.709650:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788572335.709666:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada5074a000. 00000400:00000200:3.0:1788572335.709729:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572335.709742:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:3.0:1788572335.709751:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada5176a770 00000400:00000010:3.0:1788572335.709754:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada5176a770. 00000100:00000001:3.0:1788572335.709763:0:2378:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:3.0:1788572335.709774:0:2378:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada455a4700 x1875449616093184/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572351 ref 2 fl Rpc:r/2c0/ffffffff rc 0/-1 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572335.709790:0:2378:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572335.709794:0:2378:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada455a4700 x1875449616093184/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572351 ref 2 fl Rpc:Qr/2c0/ffffffff rc 0/-1 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572335.709801:0:2378:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572335.709803:0:2378:0:(events.c:69:request_out_callback()) Process leaving 00010000:00000010:2.1F:1788572335.709888:0:1360:0:(ldlm_resource.c:1555:__ldlm_resource_free()) slab-freed '(res)': 256 at ffff9ada60762d00. 00010000:00000010:2.1:1788572335.709922:0:1360:0:(ldlm_lock.c:174:lock_handle_free()) slab-freed 'lock': 512 at ffff9ada6035ca00. 00010000:00000001:2.0:1788572335.991054:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572335.991066:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572335.991074:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572335.991079:0:365:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:2.0:1788572335.991082:0:365:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:2.0:1788572335.991097:0:365:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572335.991101:0:365:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:2.0:1788572335.991105:0:365:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572335.991106:0:365:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572335.991109:0:365:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572335.991113:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572335.991534:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572335.991703:0:15204:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572335.991707:0:15204:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572335.991710:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000800:00000001:2.0:1788572336.790899:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572336.790919:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788572336.790925:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572336.790928:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572336.790939:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572336.790950:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572336.794248:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572336.794258:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572336.794266:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572336.794276:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 4 MB=0x6a9b624295000 00000400:00000200:3.0:1788572336.794283:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572336.794289:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.202.113@tcp of length 224/224 into md 0x357e5 [1] + 224 00000800:00000001:3.0:1788572336.794301:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572336.794569:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572336.794574:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572336.794580:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572336.794591:0:2377:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572336.794602:0:2377:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada455a4700 x1875449616093184/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572351 ref 1 fl Rpc:Qr/2c0/ffffffff rc 0/-1 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572336.794614:0:2377:0:(events.c:151:reply_in_callback()) @@@ reply in flags=2c0 mlen=224 offset=224 replen=224 req@ffff9ada455a4700 x1875449616093184/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572351 ref 1 fl Rpc:RQ/2c0/ffffffff rc 0/-1 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572336.794702:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788572336.794706:0:2382:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788572336.794724:0:2377:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572336.794737:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572336.794741:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:0.0:1788572336.794809:0:2382:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788572336.794812:0:2382:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788572336.794816:0:2382:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada60acaf20. 00000400:00000200:0.0:1788572336.794822:0:2382:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada5176aaa0 00000400:00000010:0.0:1788572336.794824:0:2382:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada5176aaa0. 00000100:00000001:0.0:1788572336.794828:0:2382:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:0.0:1788572336.794833:0:2382:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada455a4700 x1875449616093184/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572351 ref 1 fl Rpc:RQ/2c0/ffffffff rc 0/-1 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788572336.794841:0:2382:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada455a4700 x1875449616093184/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572351 ref 1 fl Rpc:RQU/2c0/ffffffff rc 0/-1 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572336.794848:0:2382:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788572336.794953:0:2382:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572336.794956:0:2382:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:0.0:1788572336.794960:0:2382:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788572336.794963:0:2382:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788572336.794966:0:2382:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572336.794969:0:2382:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788572336.794981:0:2382:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada48407c60 time=31 v=5 (5 5 5 5) 00000100:00001000:0.0:1788572336.794986:0:2382:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada48407c18 time=1 v=5 (1 0 0 0) 00000100:00000001:0.0:1788572336.794988:0:2382:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788572336.794990:0:2382:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788572336.794994:0:2382:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788572336.794998:0:2382:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572336.795003:0:2382:0:(client.c:3110:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788572336.795005:0:2382:0:(client.c:3122:ptlrpc_free_committed()) lustre-OST0000-osc-ffff9ada58b0a000: skip recheck: last_committed 8589935763 00000100:00000001:0.0:1788572336.795007:0:2382:0:(client.c:3123:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788572336.795008:0:2382:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572336.795014:0:2382:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada455a4700 x1875449616093184/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572351 ref 1 fl Rpc:RQU/2c0/0 rc 0/0 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572336.795021:0:2382:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788572336.795024:0:2382:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788572336.795026:0:2382:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572336.795029:0:2382:0:(import.c:1572:completed_replay_interpret()) Process entered 00000100:00000001:0.0:1788572336.795031:0:2382:0:(import.c:1692:ptlrpc_import_recovery_state_machine()) Process entered 00000100:00080000:0.0:1788572336.795035:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada48407800 lustre-OST0000_UUID: changing import state from REPLAY_WAIT to RECOVER 00000100:00000001:0.0:1788572336.795038:0:2382:0:(recover.c:149:ptlrpc_resend()) Process entered 00000100:00080000:0.0:1788572336.795041:0:2382:0:(client.c:3248:ptlrpc_resend_req()) @@@ going to resend req@ffff9ada455a4700 x1875449616093184/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572351 ref 1 fl Interpret:RQU/2c0/0 rc 0/0 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572336.795049:0:2382:0:(client.c:3257:ptlrpc_resend_req()) @@@ it has reply, so skip it req@ffff9ada455a4700 x1875449616093184/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572351 ref 1 fl Interpret:RQU/2c0/0 rc 0/0 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572336.795055:0:2382:0:(recover.c:178:ptlrpc_resend()) Process leaving (rc=0 : 0 : 0) 00000100:00080000:0.0:1788572336.795057:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada48407800 lustre-OST0000_UUID: changing import state from RECOVER to FULL 00000100:00000001:0.0:1788572336.795059:0:2382:0:(obd_class.h:1222:obd_import_event()) Process entered 00000008:00000001:0.0:1788572336.795064:0:2382:0:(osc_request.c:3902:osc_import_event()) Process entered 00000008:00000001:0.0:1788572336.795066:0:2382:0:(obd_class.h:1242:obd_notify()) Process entered 00020000:00000001:0.0:1788572336.795070:0:2382:0:(lov_obd.c:430:lov_notify()) Process entered 00020000:00000001:1.0:1788572336.795169:0:2382:0:(lov_obd.c:349:lov_set_osc_active()) Process entered 00020000:00000040:1.0:1788572336.795173:0:2382:0:(lov_obd.c:351:lov_set_osc_active()) Searching in lov ffff9ada4481bba8 for uuid lustre-OST0000_UUID event(1) 00020000:00000040:1.0:1788572336.795179:0:2382:0:(lov_obd.c:394:lov_set_osc_active()) OSC lustre-OST0000_UUID already active! 00020000:00000001:1.0:1788572336.795182:0:2382:0:(lov_obd.c:395:lov_set_osc_active()) Process leaving via out (rc=0 : 0 : 0x0) 00020000:00000001:1.0:1788572336.795185:0:2382:0:(lov_obd.c:421:lov_set_osc_active()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:1.0:1788572336.795190:0:2382:0:(llite_lib.c:386:ll_upcall_notify()) Process entered 00000080:00000001:1.0:1788572336.795194:0:2382:0:(lcommon_misc.c:85:cl_ocd_update()) Process entered 00000080:00000004:1.0:1788572336.795196:0:2382:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0xa0427af2e3440038 -> 0xa0427af2e3440038 00000080:00000001:1.0:1788572336.795199:0:2382:0:(lcommon_misc.c:37:cl_init_ea_size()) Process entered 00000080:00000001:1.0:1788572336.795200:0:2382:0:(obd_class.h:510:obd_get_info()) Process entered 00020000:00000001:1.0:1788572336.795204:0:2382:0:(lov_obd.c:1160:lov_get_info()) Process entered 00020000:00000001:1.0:1788572336.795206:0:2382:0:(lov_obd.c:1211:lov_get_info()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:1.0:1788572336.795208:0:2382:0:(obd_class.h:524:obd_get_info()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:1.0:1788572336.795210:0:2382:0:(obd_class.h:510:obd_get_info()) Process entered 00020000:00000001:1.0:1788572336.795212:0:2382:0:(lov_obd.c:1160:lov_get_info()) Process entered 00020000:00000001:1.0:1788572336.795214:0:2382:0:(lov_obd.c:1211:lov_get_info()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:1.0:1788572336.795215:0:2382:0:(obd_class.h:524:obd_get_info()) Process leaving (rc=0 : 0 : 0) 00000080:00080000:1.0:1788572336.795217:0:2382:0:(lcommon_misc.c:56:cl_init_ea_size()) updating def/max_easize: 72/65536 00800000:00000001:1.0:1788572336.795222:0:2382:0:(lmv_obd.c:286:lmv_init_ea_size()) Process entered 00800000:00000001:1.0:1788572336.795223:0:2382:0:(lmv_obd.c:298:lmv_init_ea_size()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:1.0:1788572336.795225:0:2382:0:(lcommon_misc.c:59:cl_init_ea_size()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:1.0:1788572336.795227:0:2382:0:(lcommon_misc.c:111:cl_ocd_update()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:1.0:1788572336.795229:0:2382:0:(llite_lib.c:411:ll_upcall_notify()) Process leaving (rc=0 : 0 : 0) 00020000:00000001:1.0:1788572336.795232:0:2382:0:(lov_obd.c:467:lov_notify()) Process leaving (rc=0 : 0 : 0) 00000008:00000001:1.0:1788572336.795234:0:2382:0:(obd_class.h:1255:obd_notify()) Process leaving (rc=0 : 0 : 0) 00000008:00000001:1.0:1788572336.795237:0:2382:0:(osc_request.c:3969:osc_import_event()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572336.795239:0:2382:0:(obd_class.h:1233:obd_import_event()) Process leaving 00000100:00080000:1.0:1788572336.795243:0:2382:0:(import.c:1787:ptlrpc_import_recovery_state_machine()) lustre-OST0000-osc-ffff9ada58b0a000: Connection restored to 192.168.202.113@tcp (at 192.168.202.113@tcp) 00000100:00000001:1.0:1788572336.795247:0:2382:0:(import.c:1798:ptlrpc_import_recovery_state_machine()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572336.795249:0:2382:0:(import.c:1590:completed_replay_interpret()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572336.795254:0:2382:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada455a4700 x1875449616093184/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572351 ref 1 fl Interpret:RQU/2c0/0 rc 0/0 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572336.795263:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616093184:192.168.202.113@tcp:400:ldlm_lock_repla.0 00000100:00000001:1.0:1788572336.795269:0:2382:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572336.795272:0:2382:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788572336.795276:0:2382:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455a4700 x1875449616093184/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572351 ref 1 fl Complete:RQU/2c0/0 rc 0/0 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572336.795288:0:2382:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788572336.795290:0:2382:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788572336.795295:0:2382:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada456bb800. 02000000:00000001:1.0:1788572336.795300:0:2382:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788572336.795303:0:2382:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788572336.795304:0:2382:0:(genops.c:1223:class_import_put()) import ffff9ada48407800 refcount=2 obd=lustre-OST0000-osc-ffff9ada58b0a000 00000020:00000001:1.0:1788572336.795307:0:2382:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1788572336.795310:0:2382:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada50c31400. 02000000:00000001:1.0:1788572336.795314:0:2382:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788572336.795420:0:2382:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788572336.795425:0:2382:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455a4700. 00000100:00000001:1.0:1788572336.795429:0:2382:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788572336.795431:0:2382:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572336.795434:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572336.795444:0:2382:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788572336.795446:0:2382:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572336.795449:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572336.795450:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572337.016399:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572337.016408:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572337.016415:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572337.016417:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572337.016421:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572337.016422:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788572337.017133:0:33423:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788572337.017138:0:33423:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788572337.017142:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788572337.527845:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788572337.527890:0:58077:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788572337.527900:0:58077:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada456bfb80. 00000020:00000040:1.0:1788572337.527908:0:58077:0:(genops.c:1210:class_import_get()) import ffff9ada58a62800 refcount=4 obd=MGC192.168.202.113@tcp 00000100:00000001:1.0:1788572337.527913:0:58077:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788572337.527916:0:58077:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788572337.527922:0:58077:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788572337.527931:0:58077:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada48e6f800. 00000100:00000001:1.0:1788572337.527938:0:58077:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572337.527953:0:58077:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS req@ffff9ada456bfb80 x1875449616093312/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788572337.527965:0:58077:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788572337.527970:0:58077:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572337.527977:0:58077:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000acbb987d] to pc [ptlrpcd_00_01+1] req@ffff9ada456bfb80 x1875449616093312/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572337.528019:0:58077:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572337.528023:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788572337.528026:0:58077:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788572337.528029:0:58077:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada456bed80. 00000020:00000040:1.0:1788572337.528032:0:58077:0:(genops.c:1210:class_import_get()) import ffff9ada51140000 refcount=4 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000100:00000001:1.0:1788572337.528034:0:58077:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788572337.528036:0:58077:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788572337.528037:0:58077:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788572337.528040:0:58077:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada48e6ff00. 00000100:00000001:1.0:1788572337.528044:0:58077:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572337.528048:0:58077:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID req@ffff9ada456bed80 x1875449616093440/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788572337.528056:0:58077:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788572337.528057:0:58077:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572337.528061:0:58077:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000076fe71b4] to pc [ptlrpcd_00_00+0] req@ffff9ada456bed80 x1875449616093440/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572337.528069:0:58077:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572337.528072:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788572337.528075:0:58077:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788572337.528078:0:58077:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada456bd880. 00000020:00000040:1.0:1788572337.528080:0:58077:0:(genops.c:1210:class_import_get()) import ffff9ada58b08000 refcount=4 obd=lustre-MDT0001-mdc-ffff9ada58b0a000 00000100:00000001:1.0:1788572337.528083:0:58077:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788572337.528084:0:58077:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788572337.528086:0:58077:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788572337.528089:0:58077:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada48e6f600. 00000100:00000001:1.0:1788572337.528092:0:58077:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572337.528096:0:58077:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID req@ffff9ada456bd880 x1875449616093568/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788572337.528102:0:58077:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788572337.528105:0:58077:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572337.528109:0:58077:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000082674ba1] to pc [ptlrpcd_00_01+1] req@ffff9ada456bd880 x1875449616093568/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572337.528115:0:58077:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572337.528118:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788572337.528121:0:58077:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788572337.528125:0:58077:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada456bd180. 00000020:00000040:1.0:1788572337.528127:0:58077:0:(genops.c:1210:class_import_get()) import ffff9ada48407800 refcount=3 obd=lustre-OST0000-osc-ffff9ada58b0a000 00000100:00000001:1.0:1788572337.528129:0:58077:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788572337.528131:0:58077:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788572337.528132:0:58077:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788572337.528135:0:58077:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada48e6fd00. 00000100:00000001:1.0:1788572337.528138:0:58077:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572337.528142:0:58077:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0000_UUID req@ffff9ada456bd180 x1875449616093696/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788572337.528149:0:58077:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788572337.528151:0:58077:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572337.528155:0:58077:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000948e8606] to pc [ptlrpcd_00_00+0] req@ffff9ada456bd180 x1875449616093696/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572337.528162:0:58077:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572337.528164:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1788572337.528167:0:58077:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:1.0:1788572337.528173:0:58077:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (4260) 00000100:00000001:1.0:1788572337.528208:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572337.528212:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788572337.528215:0:2384:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788572337.528221:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada456bfb80 x1875449616093312/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572337.528229:0:2384:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788572337.528231:0:2384:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572337.528236:0:2384:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788572337.528238:0:2384:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788572337.528241:0:2384:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788572337.528245:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada456bfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449616093312:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572337.528250:0:2384:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788572337.528256:0:2384:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788572337.528259:0:2384:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572337.528262:0:2384:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788572337.528266:0:2384:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada607e1c00. 02000000:00000001:1.0:1788572337.528269:0:2384:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788572337.528279:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:1.0:1788572337.528284:0:2384:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada600d1ec8. 00000400:00000010:1.0:1788572337.528293:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d23b8. 00000100:00000200:1.0:1788572337.528303:0:2384:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616093312, portal 25 00000100:00000001:1.0:1788572337.528306:0:2384:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788572337.528307:0:2384:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860991093632 : -111212718457984 : ffff9ada456bfb80) 00000100:00000040:1.0:1788572337.528313:0:2384:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada456bfb80 x1875449616093312/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572353 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572337.528332:0:2384:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788572337.528336:0:2384:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:1.0:1788572337.528339:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2bb0. 00000100:00000200:1.0:1788572337.528343:0:2384:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875449616093312, offset 0 00000400:00000200:1.0:1788572337.528351:0:2384:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:1.0:1788572337.528362:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572337.528371:0:2384:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:1.0:1788572337.528379:0:2384:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788572337.528384:0:2384:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:1.0:1788572337.528389:0:2384:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788572337.528392:0:2384:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:1.0:1788572337.528397:0:2384:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22301:22301:256:4294967295] 192.168.202.113@tcp LPNI seq info [22297:22297:8:4294967295] 00000400:00000200:1.0:1788572337.528402:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572337.528413:0:2384:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:1.0:1788572337.528422:0:2384:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:1.0:1788572337.528428:0:2384:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada48e6f900. 00000800:00000200:1.0:1788572337.528436:0:2384:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:1.0:1788572337.528444:0:2384:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:1.0:1788572337.528449:0:2384:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada48e6f900 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788572337.528459:0:2384:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788572337.528464:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:1.0:1788572337.528467:0:2384:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS new deadline estimation for cpu 2 is 4298933068 00000100:00000001:1.0:1788572337.528474:0:2384:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4298931216 00000100:00000001:1.0:1788572337.528478:0:2384:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS new deadline estimation for cpu 3 is 4298933068 00000100:00000001:1.0:1788572337.528482:0:2384:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4298931216 00000100:00000001:1.0:1788572337.528485:0:2384:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572337.528497:0:2384:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572337.528500:0:2384:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788572337.528504:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada456bd880 x1875449616093568/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572337.528511:0:2384:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788572337.528512:0:2384:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572337.528515:0:2384:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788572337.528517:0:2384:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788572337.528519:0:2384:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788572337.528521:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada456bd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616093568:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572337.528524:0:2384:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788572337.528527:0:2384:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788572337.528528:0:2384:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572337.528530:0:2384:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788572337.528534:0:2384:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada607e0600. 02000000:00000001:1.0:1788572337.528535:0:2384:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788572337.528539:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:1.0:1788572337.528541:0:2384:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada600d1c60. 00000400:00000010:1.0:1788572337.528546:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2aa0. 00000100:00000200:1.0:1788572337.528548:0:2384:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616093568, portal 10 00000100:00000001:1.0:1788572337.528551:0:2384:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788572337.528552:0:2384:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860991084672 : -111212718466944 : ffff9ada456bd880) 00000100:00000040:1.0:1788572337.528559:0:2384:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada456bd880 x1875449616093568/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572353 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572337.528565:0:2384:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788572337.528568:0:2384:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:1.0:1788572337.528570:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2000. 00000100:00000200:1.0:1788572337.528572:0:2384:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875449616093568, offset 0 00000400:00000200:1.0:1788572337.528576:0:2384:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:1.0:1788572337.528580:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572337.528583:0:2384:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:1.0:1788572337.528586:0:2384:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788572337.528588:0:2384:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:1.0:1788572337.528591:0:2384:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788572337.528593:0:2384:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:1.0:1788572337.528596:0:2384:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22302:22302:255:4294967295] 192.168.202.113@tcp LPNI seq info [22298:22298:7:4294967295] 00000400:00000200:1.0:1788572337.528600:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572337.528606:0:2384:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:1.0:1788572337.528610:0:2384:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:1.0:1788572337.528614:0:2384:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada48e6fc00. 00000800:00000200:1.0:1788572337.528618:0:2384:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:1.0:1788572337.528623:0:2384:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:1.0:1788572337.528626:0:2384:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada48e6fc00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788572337.528628:0:2384:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572337.528633:0:2384:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: using a timeout of 16625 usecs (17 jiffies) 00000400:00000200:1.0:1788572337.528636:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:1.0:1788572337.528638:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298922513 00000100:00000001:1.0:1788572337.528640:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298933068 00000100:00000001:1.0:1788572337.528643:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298922514 00000100:00000001:1.0:1788572337.528645:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298933068 00000100:00000001:1.0:1788572337.528647:0:2384:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572337.528649:0:2384:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572337.528652:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572337.528654:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572337.528662:0:2384:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788572337.528665:0:2384:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788572337.528667:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572337.528668:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788572337.528669:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788572337.528672:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00000001:1.0:1788572337.528674:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572337.528675:0:2384:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788572337.528676:0:2384:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572337.528678:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572337.528679:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788572337.528680:0:2384:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788572337.528684:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada456bed80 x1875449616093440/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572337.528690:0:2384:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788572337.528691:0:2384:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572337.528693:0:2384:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788572337.528694:0:2384:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788572337.528696:0:2384:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788572337.528699:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada456bed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616093440:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572337.528702:0:2384:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788572337.528704:0:2384:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788572337.528706:0:2384:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572337.528708:0:2384:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788572337.528710:0:2384:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada607e0800. 02000000:00000001:1.0:1788572337.528713:0:2384:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788572337.528716:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:1.0:1788572337.528719:0:2384:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada600d16e0. 00000400:00000010:1.0:1788572337.528723:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2e58. 00000100:00000200:1.0:1788572337.528725:0:2384:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616093440, portal 10 00000100:00000001:1.0:1788572337.528727:0:2384:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788572337.528729:0:2384:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860991090048 : -111212718461568 : ffff9ada456bed80) 00000100:00000040:1.0:1788572337.528734:0:2384:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada456bed80 x1875449616093440/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572353 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572337.528740:0:2384:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788572337.528743:0:2384:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:1.0:1788572337.528745:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2660. 00000100:00000200:1.0:1788572337.528747:0:2384:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875449616093440, offset 0 00000400:00000200:1.0:1788572337.528751:0:2384:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:1.0:1788572337.528755:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572337.528758:0:2384:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:1.0:1788572337.528761:0:2384:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788572337.528763:0:2384:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:1.0:1788572337.528766:0:2384:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788572337.528768:0:2384:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:1.0:1788572337.528770:0:2384:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22303:22303:254:4294967295] 192.168.202.113@tcp LPNI seq info [22299:22299:6:4294967295] 00000400:00000200:1.0:1788572337.528774:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572337.528781:0:2384:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:1.0:1788572337.528785:0:2384:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:1.0:1788572337.528789:0:2384:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada48e6f100. 00000800:00000200:1.0:1788572337.528792:0:2384:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:1.0:1788572337.528797:0:2384:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:1.0:1788572337.528800:0:2384:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada48e6f100 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788572337.528803:0:2384:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572337.528805:0:2384:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: using a timeout of 41268 usecs (42 jiffies) 00000400:00000200:1.0:1788572337.528809:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:1.0:1788572337.528811:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298922539 00000100:00000001:1.0:1788572337.528813:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298933068 00000100:00000001:1.0:1788572337.528815:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298922539 00000100:00000001:1.0:1788572337.528817:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298933068 00000100:00000001:1.0:1788572337.528819:0:2384:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572337.528821:0:2384:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572337.528823:0:2384:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788572337.528827:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada456bd180 x1875449616093696/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572337.528833:0:2384:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788572337.528834:0:2384:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572337.528836:0:2384:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788572337.528837:0:2384:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788572337.528839:0:2384:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788572337.528841:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada456bd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616093696:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572337.528844:0:2384:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788572337.528846:0:2384:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788572337.528847:0:2384:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572337.528849:0:2384:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788572337.528880:0:2384:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada607e0c00. 02000000:00000001:1.0:1788572337.528882:0:2384:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788572337.528886:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:1.0:1788572337.528890:0:2384:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada600d1bb0. 00000400:00000010:1.0:1788572337.528893:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d22a8. 00000100:00000200:1.0:1788572337.528897:0:2384:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616093696, portal 4 00000100:00000001:1.0:1788572337.528898:0:2384:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788572337.528900:0:2384:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860991082880 : -111212718468736 : ffff9ada456bd180) 00000100:00000040:1.0:1788572337.528906:0:2384:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada456bd180 x1875449616093696/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572353 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572337.528912:0:2384:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788572337.528915:0:2384:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:1.0:1788572337.528917:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2d48. 00000100:00000200:1.0:1788572337.528919:0:2384:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1875449616093696, offset 0 00000400:00000200:1.0:1788572337.528924:0:2384:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:1.0:1788572337.528928:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572337.528931:0:2384:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:1.0:1788572337.528933:0:2384:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788572337.528936:0:2384:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:1.0:1788572337.528938:0:2384:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788572337.528940:0:2384:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:1.0:1788572337.528943:0:2384:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22304:22304:253:4294967295] 192.168.202.113@tcp LPNI seq info [22300:22300:5:4294967295] 00000400:00000200:1.0:1788572337.528947:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572337.528952:0:2384:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:1.0:1788572337.528957:0:2384:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:1.0:1788572337.528960:0:2384:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada48e6f400. 00000800:00000200:1.0:1788572337.528963:0:2384:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:1.0:1788572337.528967:0:2384:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:1.0:1788572337.528970:0:2384:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada48e6f400 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788572337.528972:0:2384:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572337.528975:0:2384:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: using a timeout of 19828 usecs (20 jiffies) 00000400:00000200:1.0:1788572337.528977:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:1.0:1788572337.528979:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298922518 00000100:00000001:1.0:1788572337.528981:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298933068 00000100:00000001:1.0:1788572337.528983:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298922518 00000100:00000001:1.0:1788572337.528985:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298933068 00000100:00000001:1.0:1788572337.528987:0:2384:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572337.528990:0:2384:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572337.528992:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572337.528994:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572337.529005:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572337.529007:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788572337.537762:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788572337.537777:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada48e6f900. 00000400:00000200:3.0:1788572337.537784:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572337.537795:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:3.0:1788572337.537803:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2bb0 00000400:00000010:3.0:1788572337.537806:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2bb0. 00000100:00000001:3.0:1788572337.537814:0:2378:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:3.0:1788572337.537825:0:2378:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada456bfb80 x1875449616093312/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572353 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572337.537839:0:2378:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572337.537842:0:2378:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada456bfb80 x1875449616093312/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572353 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572337.537849:0:2378:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572337.537953:0:2378:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000200:3.0:1788572337.537971:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788572337.537974:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada48e6fc00. 00000400:00000200:3.0:1788572337.537977:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572337.537981:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:3.0:1788572337.537984:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2000 00000400:00000010:3.0:1788572337.537986:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2000. 00000100:00000001:3.0:1788572337.537988:0:2378:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:3.0:1788572337.537992:0:2378:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada456bd880 x1875449616093568/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572353 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572337.538000:0:2378:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572337.538003:0:2378:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada456bd880 x1875449616093568/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572353 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572337.538085:0:2378:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572337.538087:0:2378:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000200:3.0:1788572337.538098:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788572337.538101:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada48e6f100. 00000400:00000200:3.0:1788572337.538104:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572337.538108:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:3.0:1788572337.538111:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2660 00000400:00000010:3.0:1788572337.538113:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2660. 00000100:00000001:3.0:1788572337.538116:0:2378:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:3.0:1788572337.538120:0:2378:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada456bed80 x1875449616093440/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572353 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572337.538127:0:2378:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572337.538129:0:2378:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada456bed80 x1875449616093440/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572353 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572337.538135:0:2378:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572337.538137:0:2378:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000200:3.0:1788572337.538448:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788572337.538452:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada48e6f400. 00000400:00000200:3.0:1788572337.538455:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572337.538460:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:3.0:1788572337.538465:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2d48 00000400:00000010:3.0:1788572337.538467:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2d48. 00000100:00000001:3.0:1788572337.538470:0:2378:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:3.0:1788572337.538475:0:2378:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada456bd180 x1875449616093696/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572353 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572337.538483:0:2378:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572337.538486:0:2378:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada456bd180 x1875449616093696/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572353 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572337.538550:0:2378:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572337.538552:0:2378:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000001:3.0:1788572337.546782:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572337.546806:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572337.546811:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572337.546814:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572337.546825:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572337.546834:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572337.546844:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572337.546850:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572337.546912:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572337.546917:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 10 MB=0x6a9b624295180 00000400:00000200:3.0:1788572337.546923:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572337.546931:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.113@tcp of length 224/224 into md 0x357f5 [1] + 224 00000800:00000001:3.0:1788572337.546940:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572337.546945:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572337.546950:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572337.546955:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572337.546967:0:2377:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572337.546979:0:2377:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada456bd880 x1875449616093568/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572353 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572337.546992:0:2377:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada456bd880 x1875449616093568/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572353 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572337.547639:0:2377:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572337.547654:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572337.547662:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572337.547666:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572337.547668:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572337.547674:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572337.547679:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572337.547686:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572337.547689:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572337.547692:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572337.547695:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 10 MB=0x6a9b624295100 00000400:00000200:3.0:1788572337.547699:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572337.547704:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.113@tcp of length 224/224 into md 0x357fd [1] + 224 00000800:00000001:3.0:1788572337.547709:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572337.547713:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572337.547715:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572337.547719:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572337.547723:0:2377:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572337.547728:0:2377:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada456bed80 x1875449616093440/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572353 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572337.547737:0:2377:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada456bed80 x1875449616093440/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572353 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572337.547745:0:2377:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572337.547749:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572337.547753:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572337.547754:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572337.547756:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572337.547759:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572337.547763:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572337.547767:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572337.547770:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572337.547772:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572337.547775:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 25 MB=0x6a9b624295080 00000400:00000200:3.0:1788572337.547779:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572337.547784:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.202.113@tcp of length 224/224 into md 0x357ed [1] + 224 00000800:00000001:3.0:1788572337.547787:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572337.547790:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572337.547792:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572337.547795:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572337.547797:0:2377:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572337.547802:0:2377:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada456bfb80 x1875449616093312/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572353 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572337.547810:0:2377:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada456bfb80 x1875449616093312/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572353 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572337.547817:0:2377:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572337.547820:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572337.547823:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572337.547824:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572337.547826:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572337.547829:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572337.547832:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572337.547836:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572337.547838:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572337.547841:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572337.547843:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 4 MB=0x6a9b624295200 00000400:00000200:3.0:1788572337.547847:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572337.547900:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.202.113@tcp of length 224/224 into md 0x35805 [1] + 224 00000800:00000001:3.0:1788572337.547905:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572337.547914:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572337.547916:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572337.547920:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572337.547923:0:2377:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572337.547928:0:2377:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada456bd180 x1875449616093696/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572353 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572337.547937:0:2377:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada456bd180 x1875449616093696/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572353 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572337.547944:0:2377:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572337.547948:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572337.547951:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572337.549172:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572337.549183:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788572337.549193:0:2384:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788572337.549197:0:2384:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788572337.549201:0:2384:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada600d16e0. 00000400:00000200:1.0:1788572337.549211:0:2384:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2e58 00000400:00000010:1.0:1788572337.549214:0:2384:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2e58. 00000100:00000001:1.0:1788572337.549221:0:2384:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:1.0:1788572337.549232:0:2384:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada456bed80 x1875449616093440/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572353 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788572337.549244:0:2384:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada456bed80 x1875449616093440/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572353 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572337.549250:0:2384:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788572337.549253:0:2384:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572337.549255:0:2384:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1788572337.549259:0:2384:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788572337.549262:0:2384:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788572337.549264:0:2384:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572337.549268:0:2384:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788572337.549280:0:2384:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada51140460 time=32 v=5 (5 5 5 5) 00000100:00001000:1.0:1788572337.549285:0:2384:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada51140418 time=128 v=5 (2 0 0 0) 00000100:00000001:1.0:1788572337.549287:0:2384:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788572337.549289:0:2384:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572337.549293:0:2384:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788572337.549302:0:2384:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000010:1.0:1788572337.549311:0:2384:0:(ldlm_lockd.c:2249:ldlm_bl_to_thread()) kmalloced '(blwi)': 192 at ffff9ada60aed240. 00010000:00000001:1.0:1788572337.549315:0:2384:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) Process entered 00010000:00010000:1.0:1788572337.549344:0:2384:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 0/0 locks to regular blp list, 1 blwis in pool 00010000:00000001:1.0:1788572337.549347:0:2384:0:(ldlm_lockd.c:2198:__ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572337.549349:0:2384:0:(ldlm_lockd.c:2254:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572337.549351:0:2384:0:(ldlm_request.c:1723:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572337.549355:0:2384:0:(client.c:3110:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788572337.549357:0:2384:0:(client.c:3122:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: skip recheck: last_committed 219043334508 00000100:00000001:1.0:1788572337.549359:0:2384:0:(client.c:3123:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788572337.549361:0:2384:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572337.549366:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada456bed80 x1875449616093440/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572353 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572337.549373:0:2384:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572337.549377:0:2384:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788572337.549380:0:2384:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788572337.549384:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada456bed80 x1875449616093440/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572353 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572337.549391:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada456bed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616093440:192.168.202.113@tcp:400:kworker.0 00000400:00000001:1.0:1788572337.549398:0:2384:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788572337.549400:0:2384:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788572337.549402:0:2384:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada600d1bb0. 00000400:00000200:1.0:1788572337.549404:0:2384:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d22a8 00000400:00000010:1.0:1788572337.549406:0:2384:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d22a8. 00000100:00000001:1.0:1788572337.549409:0:2384:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:1.0:1788572337.549413:0:2384:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada456bd180 x1875449616093696/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572353 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788572337.549422:0:2384:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada456bd180 x1875449616093696/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572353 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572337.549428:0:2384:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788572337.549430:0:2384:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572337.549432:0:2384:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1788572337.549433:0:2384:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788572337.549434:0:2384:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788572337.549436:0:2384:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572337.549438:0:2384:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788572337.549442:0:2384:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada48407c60 time=32 v=5 (5 5 5 5) 00000100:00001000:1.0:1788572337.549445:0:2384:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada48407c18 time=2 v=5 (1 0 0 0) 00000100:00000001:1.0:1788572337.549448:0:2384:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788572337.549449:0:2384:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572337.549450:0:2384:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788572337.549452:0:2384:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572337.549455:0:2384:0:(client.c:3110:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788572337.549456:0:2384:0:(client.c:3122:ptlrpc_free_committed()) lustre-OST0000-osc-ffff9ada58b0a000: skip recheck: last_committed 8589935763 00000100:00000001:1.0:1788572337.549458:0:2384:0:(client.c:3123:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788572337.549460:0:2384:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572337.549463:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada456bd180 x1875449616093696/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572353 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572337.549470:0:2384:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572337.549472:0:2384:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788572337.549473:0:2384:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000020:00000001:2.0:1788572337.549475:0:15205:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000100:00000040:1.0:1788572337.549477:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada456bd180 x1875449616093696/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572353 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000020:00000001:2.0:1788572337.549479:0:15205:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788572337.549482:0:15205:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 0 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:1.0:1788572337.549484:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada456bd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616093696:192.168.202.113@tcp:400:kworker.0 00010000:00000001:2.0:1788572337.549486:0:15205:0:(ldlm_lockd.c:2881:ldlm_bl_thread_blwi()) Process entered 00010000:00000001:2.0:1788572337.549492:0:15205:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572337.549495:0:15205:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:2.0:1788572337.549498:0:15205:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00000400:00000001:1.0:1788572337.549499:0:2384:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788572337.549500:0:2384:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788572337.549502:0:2384:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada600d1ec8. 00000400:00000200:1.0:1788572337.549505:0:2384:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d23b8 00010000:00000001:2.0:1788572337.549505:0:15205:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00000400:00000010:1.0:1788572337.549507:0:2384:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d23b8. 00010000:00000001:2.0:1788572337.549507:0:15205:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00000100:00000001:1.0:1788572337.549509:0:2384:0:(events.c:80:reply_in_callback()) Process entered 00010000:00000001:2.0:1788572337.549510:0:15205:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572337.549512:0:15205:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00000100:00000200:1.0:1788572337.549513:0:2384:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada456bfb80 x1875449616093312/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572353 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00000001:2.0:1788572337.549514:0:15205:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572337.549516:0:15205:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572337.549519:0:15205:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1788572337.549521:0:2384:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada456bfb80 x1875449616093312/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572353 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00000010:2.0:1788572337.549523:0:15205:0:(ldlm_lockd.c:2919:ldlm_bl_thread_blwi()) kfreed 'blwi': 192 at ffff9ada60aed240. 00000100:00000001:1.0:1788572337.549527:0:2384:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788572337.549529:0:2384:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:2.0:1788572337.549529:0:15205:0:(ldlm_lockd.c:2923:ldlm_bl_thread_blwi()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572337.549531:0:2384:0:(client.c:1593:after_reply()) Process entered 00000020:00000001:2.0:1788572337.549531:0:15205:0:(genops.c:1895:obd_stale_export_get()) Process entered 02000000:00000001:1.0:1788572337.549532:0:2384:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000020:00000001:2.0:1788572337.549533:0:15205:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572337.549534:0:2384:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788572337.549535:0:2384:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788572337.549535:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 02000000:00000001:1.0:1788572337.549537:0:2384:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788572337.549537:0:15205:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788572337.549538:0:15205:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788572337.549540:0:2384:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada58a62c60 time=99 v=5 (5 5 5 5) 00010000:00010000:2.0:1788572337.549540:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000001:2.0:1788572337.549542:0:15205:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000100:00001000:1.0:1788572337.549543:0:2384:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada58a62c18 time=129 v=5 (1 0 0 0) 00000020:00000001:2.0:1788572337.549543:0:15205:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788572337.549545:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000001:1.0:1788572337.549546:0:2384:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788572337.549547:0:2384:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572337.549549:0:2384:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788572337.549550:0:2384:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572337.549552:0:2384:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572337.549555:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada456bfb80 x1875449616093312/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572353 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572337.549561:0:2384:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572337.549563:0:2384:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788572337.549564:0:2384:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788572337.549567:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada456bfb80 x1875449616093312/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572353 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572337.549574:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada456bfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449616093312:192.168.202.113@tcp:400:kworker.0 00000400:00000001:1.0:1788572337.549578:0:2384:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788572337.549580:0:2384:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788572337.549582:0:2384:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada600d1c60. 00000400:00000200:1.0:1788572337.549584:0:2384:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2aa0 00000400:00000010:1.0:1788572337.549586:0:2384:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2aa0. 00000100:00000001:1.0:1788572337.549588:0:2384:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:1.0:1788572337.549592:0:2384:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada456bd880 x1875449616093568/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572353 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788572337.549599:0:2384:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada456bd880 x1875449616093568/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572353 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572337.549604:0:2384:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788572337.549605:0:2384:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572337.549606:0:2384:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1788572337.549608:0:2384:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788572337.549609:0:2384:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788572337.549610:0:2384:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572337.549612:0:2384:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788572337.549616:0:2384:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada58b08460 time=99 v=5 (5 5 5 5) 00000100:00001000:1.0:1788572337.549619:0:2384:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada58b08418 time=99 v=5 (1 1 1 1) 00000100:00000001:1.0:1788572337.549621:0:2384:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788572337.549622:0:2384:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572337.549623:0:2384:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788572337.549627:0:2384:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000010:1.0:1788572337.549630:0:2384:0:(ldlm_lockd.c:2249:ldlm_bl_to_thread()) kmalloced '(blwi)': 192 at ffff9ada60aedd80. 00010000:00000001:1.0:1788572337.549632:0:2384:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) Process entered 00010000:00010000:1.0:1788572337.549646:0:2384:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 0/0 locks to regular blp list, 1 blwis in pool 00010000:00000001:1.0:1788572337.549648:0:2384:0:(ldlm_lockd.c:2198:__ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572337.549650:0:2384:0:(ldlm_lockd.c:2254:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572337.549651:0:2384:0:(ldlm_request.c:1723:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572337.549653:0:2384:0:(client.c:3110:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788572337.549654:0:2384:0:(client.c:3122:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff9ada58b0a000: skip recheck: last_committed 0 00000100:00000001:1.0:1788572337.549656:0:2384:0:(client.c:3123:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788572337.549657:0:2384:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572337.549661:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada456bd880 x1875449616093568/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572353 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572337.549667:0:2384:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572337.549669:0:2384:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788572337.549670:0:2384:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788572337.549673:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada456bd880 x1875449616093568/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572353 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572337.549679:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada456bd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616093568:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572337.549684:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572337.549687:0:2384:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788572337.549690:0:2384:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada456bed80 x1875449616093440/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572353 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572337.549697:0:2384:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788572337.549699:0:2384:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788572337.549702:0:2384:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada607e0800. 02000000:00000001:1.0:1788572337.549705:0:2384:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788572337.549708:0:2384:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788572337.549709:0:2384:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=3 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:1.0:1788572337.549711:0:2384:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1788572337.549714:0:2384:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada48e6ff00. 02000000:00000001:1.0:1788572337.549718:0:2384:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788572337.549720:0:2384:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788572337.549722:0:2384:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada456bed80. 00000100:00000001:1.0:1788572337.549726:0:2384:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788572337.549729:0:2384:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572337.549730:0:2384:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788572337.549734:0:2384:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada456bd180 x1875449616093696/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572353 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572337.549740:0:2384:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788572337.549742:0:2384:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788572337.549744:0:2384:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada607e0c00. 02000000:00000001:1.0:1788572337.549746:0:2384:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788572337.549748:0:2384:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788572337.549749:0:2384:0:(genops.c:1223:class_import_put()) import ffff9ada48407800 refcount=2 obd=lustre-OST0000-osc-ffff9ada58b0a000 00000020:00000001:1.0:1788572337.549751:0:2384:0:(genops.c:1230:class_import_put()) Process leaving 00000020:00000001:2.0:1788572337.549752:0:15204:0:(genops.c:1895:obd_stale_export_get()) Process entered 02000000:00000010:1.0:1788572337.549753:0:2384:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada48e6fd00. 00000020:00000001:2.0:1788572337.549754:0:15204:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572337.549755:0:2384:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788572337.549756:0:2384:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00010000:00010000:2.0:1788572337.549756:0:15204:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 0 locks of 0 total in blp. (0 blwis in pool) 00000100:00000010:1.0:1788572337.549758:0:2384:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada456bd180. 00010000:00000001:2.0:1788572337.549758:0:15204:0:(ldlm_lockd.c:2881:ldlm_bl_thread_blwi()) Process entered 00000100:00000001:1.0:1788572337.549760:0:2384:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788572337.549761:0:2384:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:2.0:1788572337.549761:0:15204:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00000100:00000001:1.0:1788572337.549763:0:2384:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00010000:00000001:2.0:1788572337.549763:0:15204:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:2.0:1788572337.549765:0:15204:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00000100:00000040:1.0:1788572337.549767:0:2384:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada456bfb80 x1875449616093312/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572353 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00000001:2.0:1788572337.549767:0:15204:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572337.549769:0:15204:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:2.0:1788572337.549771:0:15204:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572337.549773:0:2384:0:(client.c:2794:__ptlrpc_free_req()) Process entered 00010000:00000001:2.0:1788572337.549773:0:15204:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572337.549775:0:2384:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 00010000:00000001:2.0:1788572337.549775:0:15204:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788572337.549776:0:2384:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada607e1c00. 00010000:00000010:2.0:1788572337.549777:0:15204:0:(ldlm_lockd.c:2919:ldlm_bl_thread_blwi()) kfreed 'blwi': 192 at ffff9ada60aedd80. 02000000:00000001:1.0:1788572337.549778:0:2384:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788572337.549780:0:2384:0:(genops.c:1217:class_import_put()) Process entered 00010000:00000001:2.0:1788572337.549780:0:15204:0:(ldlm_lockd.c:2923:ldlm_bl_thread_blwi()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:1.0:1788572337.549781:0:2384:0:(genops.c:1223:class_import_put()) import ffff9ada58a62800 refcount=3 obd=MGC192.168.202.113@tcp 00000020:00000001:2.0:1788572337.549781:0:15204:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788572337.549783:0:2384:0:(genops.c:1230:class_import_put()) Process leaving 00000020:00000001:2.0:1788572337.549783:0:15204:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788572337.549785:0:2384:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada48e6f800. 00010000:00010000:2.0:1788572337.549785:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000001:2.0:1788572337.549786:0:15204:0:(genops.c:1895:obd_stale_export_get()) Process entered 02000000:00000001:1.0:1788572337.549787:0:2384:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788572337.549788:0:2384:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000020:00000001:2.0:1788572337.549788:0:15204:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:1.0:1788572337.549790:0:2384:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada456bfb80. 00010000:00010000:2.0:1788572337.549790:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000001:2.0:1788572337.549791:0:15204:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000100:00000001:1.0:1788572337.549792:0:2384:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000020:00000001:2.0:1788572337.549792:0:15204:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572337.549793:0:2384:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572337.549794:0:2384:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00010000:00010000:2.0:1788572337.549794:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788572337.549797:0:2384:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada456bd880 x1875449616093568/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572353 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572337.549809:0:2384:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788572337.549811:0:2384:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788572337.549812:0:2384:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada607e0600. 02000000:00000001:1.0:1788572337.549814:0:2384:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788572337.549815:0:2384:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788572337.549817:0:2384:0:(genops.c:1223:class_import_put()) import ffff9ada58b08000 refcount=3 obd=lustre-MDT0001-mdc-ffff9ada58b0a000 00000020:00000001:1.0:1788572337.549818:0:2384:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1788572337.549820:0:2384:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada48e6f600. 02000000:00000001:1.0:1788572337.549821:0:2384:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788572337.549822:0:2384:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788572337.549824:0:2384:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada456bd880. 00000100:00000001:1.0:1788572337.549826:0:2384:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788572337.549827:0:2384:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572337.549829:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572337.549840:0:2384:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788572337.549842:0:2384:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572337.549844:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572337.549847:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572338.039500:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572338.039519:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572338.039531:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572338.039532:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572338.039535:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572338.039536:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788572338.039753:0:33423:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788572338.039757:0:33423:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788572338.039760:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572339.063942:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572339.063952:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572339.063962:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572339.063964:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572339.063967:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572339.063968:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788572339.064014:0:15205:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788572339.064018:0:15205:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788572339.064022:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572340.087122:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572340.087131:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572340.087139:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572340.087140:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572340.087144:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572340.087145:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788572340.087175:0:15204:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788572340.087178:0:15204:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788572340.087182:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000001:2.0:1788572340.091064:0:15197:0:(service.c:1710:ptlrpc_at_check_timed()) Process entered 00000100:00000001:2.0:1788572340.091076:0:15197:0:(service.c:1721:ptlrpc_at_check_timed()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572341.111512:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572341.111521:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572341.111528:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572341.111529:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572341.111532:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572341.111533:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788572341.120582:0:33423:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788572341.120601:0:33423:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788572341.120607:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572342.157839:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572342.157850:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572342.158132:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572342.158134:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572342.158137:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572342.158138:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572342.158836:0:15205:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572342.158841:0:15205:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572342.158845:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788572342.648938:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788572342.648950:0:58077:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788572342.648960:0:58077:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada456bd880. 00000020:00000040:1.0:1788572342.648968:0:58077:0:(genops.c:1210:class_import_get()) import ffff9ada58a62800 refcount=4 obd=MGC192.168.202.113@tcp 00000100:00000001:1.0:1788572342.648973:0:58077:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788572342.648977:0:58077:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788572342.648983:0:58077:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788572342.648993:0:58077:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada48e6f600. 00000100:00000001:1.0:1788572342.649002:0:58077:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572342.649029:0:58077:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS req@ffff9ada456bd880 x1875449616093824/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788572342.649042:0:58077:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788572342.649047:0:58077:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572342.649054:0:58077:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000082674ba1] to pc [ptlrpcd_00_01+1] req@ffff9ada456bd880 x1875449616093824/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572342.649088:0:58077:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572342.649091:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788572342.649095:0:58077:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788572342.649098:0:58077:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada456bfb80. 00000020:00000040:1.0:1788572342.649102:0:58077:0:(genops.c:1210:class_import_get()) import ffff9ada51140000 refcount=4 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000100:00000001:1.0:1788572342.649104:0:58077:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788572342.649105:0:58077:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788572342.649107:0:58077:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788572342.649110:0:58077:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada48e6f800. 00000100:00000001:1.0:1788572342.649113:0:58077:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572342.649117:0:58077:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID req@ffff9ada456bfb80 x1875449616093952/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788572342.649123:0:58077:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788572342.649125:0:58077:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572342.649128:0:58077:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000acbb987d] to pc [ptlrpcd_00_00+0] req@ffff9ada456bfb80 x1875449616093952/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572342.649136:0:58077:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572342.649138:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788572342.649141:0:58077:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788572342.649144:0:58077:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada456bd180. 00000020:00000040:1.0:1788572342.649146:0:58077:0:(genops.c:1210:class_import_get()) import ffff9ada58b08000 refcount=4 obd=lustre-MDT0001-mdc-ffff9ada58b0a000 00000100:00000001:1.0:1788572342.649148:0:58077:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788572342.649149:0:58077:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788572342.649151:0:58077:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788572342.649154:0:58077:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada48e6fd00. 00000100:00000001:1.0:1788572342.649156:0:58077:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572342.649160:0:58077:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID req@ffff9ada456bd180 x1875449616094080/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788572342.649166:0:58077:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788572342.649169:0:58077:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572342.649172:0:58077:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000948e8606] to pc [ptlrpcd_00_01+1] req@ffff9ada456bd180 x1875449616094080/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572342.649180:0:58077:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572342.649182:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788572342.649185:0:58077:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788572342.649189:0:58077:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada456bed80. 00000020:00000040:1.0:1788572342.649191:0:58077:0:(genops.c:1210:class_import_get()) import ffff9ada48407800 refcount=3 obd=lustre-OST0000-osc-ffff9ada58b0a000 00000100:00000001:1.0:1788572342.649193:0:58077:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788572342.649195:0:58077:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788572342.649196:0:58077:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788572342.649199:0:58077:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada48e6ff00. 00000100:00000001:1.0:1788572342.649201:0:58077:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572342.649205:0:58077:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0000_UUID req@ffff9ada456bed80 x1875449616094208/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788572342.649211:0:58077:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788572342.649213:0:58077:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572342.649216:0:58077:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000076fe71b4] to pc [ptlrpcd_00_00+0] req@ffff9ada456bed80 x1875449616094208/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572342.649222:0:58077:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572342.649225:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1788572342.649228:0:58077:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:1.0:1788572342.649235:0:58077:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (4265) 00000100:00000001:1.0:1788572342.649249:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572342.649265:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788572342.649268:0:2384:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788572342.649274:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada456bd880 x1875449616093824/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572342.649281:0:2384:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788572342.649283:0:2384:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572342.649288:0:2384:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788572342.649291:0:2384:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788572342.649294:0:2384:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788572342.649297:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada456bd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449616093824:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572342.649302:0:2384:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788572342.649309:0:2384:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788572342.649312:0:2384:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572342.649315:0:2384:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788572342.649320:0:2384:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada607e1200. 02000000:00000001:1.0:1788572342.649322:0:2384:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788572342.649332:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:1.0:1788572342.649336:0:2384:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada600d1c60. 00000400:00000010:1.0:1788572342.649345:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2aa0. 00000100:00000200:1.0:1788572342.649355:0:2384:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616093824, portal 25 00000100:00000001:1.0:1788572342.649358:0:2384:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788572342.649360:0:2384:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860991084672 : -111212718466944 : ffff9ada456bd880) 00000100:00000040:1.0:1788572342.649366:0:2384:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada456bd880 x1875449616093824/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572358 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572342.649374:0:2384:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788572342.649377:0:2384:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:1.0:1788572342.649379:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d23b8. 00000100:00000200:1.0:1788572342.649383:0:2384:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875449616093824, offset 0 00000400:00000200:1.0:1788572342.649390:0:2384:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:1.0:1788572342.649401:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572342.649411:0:2384:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:1.0:1788572342.649418:0:2384:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788572342.649423:0:2384:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:1.0:1788572342.649429:0:2384:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788572342.649431:0:2384:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:1.0:1788572342.649436:0:2384:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22305:22305:256:4294967295] 192.168.202.113@tcp LPNI seq info [22301:22301:8:4294967295] 00000400:00000200:1.0:1788572342.649441:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572342.649453:0:2384:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:1.0:1788572342.649462:0:2384:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:1.0:1788572342.649468:0:2384:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada48e6fa00. 00000800:00000200:1.0:1788572342.649475:0:2384:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:1.0:1788572342.649484:0:2384:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:1.0:1788572342.649491:0:2384:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada48e6fa00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788572342.649504:0:2384:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788572342.649509:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:1.0:1788572342.649512:0:2384:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS new deadline estimation for cpu 2 is 4298938190 00000100:00000001:1.0:1788572342.649520:0:2384:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4298933068 00000100:00000001:1.0:1788572342.649525:0:2384:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS new deadline estimation for cpu 3 is 4298938190 00000100:00000001:1.0:1788572342.649529:0:2384:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4298933068 00000100:00000001:1.0:1788572342.649531:0:2384:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572342.649534:0:2384:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572342.649537:0:2384:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788572342.649541:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada456bd180 x1875449616094080/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572342.649547:0:2384:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788572342.649548:0:2384:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572342.649550:0:2384:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788572342.649553:0:2384:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788572342.649554:0:2384:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788572342.649556:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada456bd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616094080:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572342.649559:0:2384:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788572342.649562:0:2384:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788572342.649563:0:2384:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572342.649565:0:2384:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788572342.649568:0:2384:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada607e0a00. 02000000:00000001:1.0:1788572342.649570:0:2384:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788572342.649573:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:1.0:1788572342.649576:0:2384:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada600d1ec8. 00000400:00000010:1.0:1788572342.649580:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d22a8. 00000100:00000200:1.0:1788572342.649583:0:2384:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616094080, portal 10 00000100:00000001:1.0:1788572342.649584:0:2384:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788572342.649585:0:2384:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860991082880 : -111212718468736 : ffff9ada456bd180) 00000100:00000040:1.0:1788572342.649592:0:2384:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada456bd180 x1875449616094080/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572358 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572342.649598:0:2384:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788572342.649600:0:2384:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:1.0:1788572342.649602:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2e58. 00000100:00000200:1.0:1788572342.649604:0:2384:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875449616094080, offset 0 00000400:00000200:1.0:1788572342.649609:0:2384:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:1.0:1788572342.649613:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572342.649616:0:2384:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:1.0:1788572342.649619:0:2384:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788572342.649621:0:2384:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:1.0:1788572342.649624:0:2384:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788572342.649626:0:2384:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:1.0:1788572342.649628:0:2384:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22306:22306:255:4294967295] 192.168.202.113@tcp LPNI seq info [22302:22302:7:4294967295] 00000400:00000200:1.0:1788572342.649632:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572342.649758:0:2384:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:1.0:1788572342.649764:0:2384:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:1.0:1788572342.649769:0:2384:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada48e6f400. 00000800:00000200:1.0:1788572342.649773:0:2384:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:1.0:1788572342.649778:0:2384:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:1.0:1788572342.649781:0:2384:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada48e6f400 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788572342.649783:0:2384:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572342.649787:0:2384:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: using a timeout of 16636 usecs (17 jiffies) 00000400:00000200:1.0:1788572342.649791:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:1.0:1788572342.649793:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298927635 00000100:00000001:1.0:1788572342.649796:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298938190 00000100:00000001:1.0:1788572342.649798:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298927635 00000100:00000001:1.0:1788572342.649801:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298938190 00000100:00000001:1.0:1788572342.649804:0:2384:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572342.649806:0:2384:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572342.649809:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572342.649812:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572342.649821:0:2384:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788572342.649824:0:2384:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788572342.649826:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572342.649828:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788572342.649829:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788572342.649832:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00000001:1.0:1788572342.649833:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572342.649835:0:2384:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788572342.649836:0:2384:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572342.649838:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572342.649839:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788572342.649840:0:2384:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788572342.649844:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada456bfb80 x1875449616093952/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572342.649850:0:2384:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788572342.649889:0:2384:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572342.649892:0:2384:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788572342.649894:0:2384:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788572342.649896:0:2384:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788572342.649899:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada456bfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616093952:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572342.649903:0:2384:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788572342.649906:0:2384:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788572342.649907:0:2384:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572342.649909:0:2384:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788572342.649912:0:2384:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada607e1e00. 02000000:00000001:1.0:1788572342.649915:0:2384:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788572342.649918:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:1.0:1788572342.649922:0:2384:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada600d1bb0. 00000400:00000010:1.0:1788572342.649926:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d25d8. 00000100:00000200:1.0:1788572342.649929:0:2384:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616093952, portal 10 00000100:00000001:1.0:1788572342.649932:0:2384:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788572342.649933:0:2384:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860991093632 : -111212718457984 : ffff9ada456bfb80) 00000100:00000040:1.0:1788572342.649939:0:2384:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada456bfb80 x1875449616093952/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572358 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572342.649946:0:2384:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788572342.649948:0:2384:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:1.0:1788572342.649950:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2ee0. 00000100:00000200:1.0:1788572342.649953:0:2384:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875449616093952, offset 0 00000400:00000200:1.0:1788572342.649958:0:2384:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:1.0:1788572342.649962:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572342.649965:0:2384:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:1.0:1788572342.649967:0:2384:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788572342.649970:0:2384:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:1.0:1788572342.649972:0:2384:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788572342.649974:0:2384:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:1.0:1788572342.649977:0:2384:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22307:22307:254:4294967295] 192.168.202.113@tcp LPNI seq info [22303:22303:6:4294967295] 00000400:00000200:1.0:1788572342.649980:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572342.649986:0:2384:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:1.0:1788572342.649990:0:2384:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:1.0:1788572342.649994:0:2384:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada48e6f100. 00000800:00000200:1.0:1788572342.649998:0:2384:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:1.0:1788572342.650002:0:2384:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:1.0:1788572342.650004:0:2384:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada48e6f100 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788572342.650019:0:2384:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572342.650022:0:2384:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: using a timeout of 41267 usecs (42 jiffies) 00000400:00000200:1.0:1788572342.650026:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:1.0:1788572342.650029:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298927661 00000100:00000001:1.0:1788572342.650031:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298938190 00000100:00000001:1.0:1788572342.650034:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298927661 00000100:00000001:1.0:1788572342.650036:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298938190 00000100:00000001:1.0:1788572342.650038:0:2384:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572342.650040:0:2384:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572342.650042:0:2384:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788572342.650046:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada456bed80 x1875449616094208/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572342.650053:0:2384:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788572342.650054:0:2384:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572342.650056:0:2384:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788572342.650058:0:2384:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788572342.650060:0:2384:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788572342.650062:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada456bed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616094208:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572342.650066:0:2384:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788572342.650068:0:2384:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788572342.650070:0:2384:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572342.650071:0:2384:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788572342.650074:0:2384:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada607e1800. 02000000:00000001:1.0:1788572342.650076:0:2384:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788572342.650079:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:1.0:1788572342.650081:0:2384:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada600d16e0. 00000400:00000010:1.0:1788572342.650085:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2088. 00000100:00000200:1.0:1788572342.650088:0:2384:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616094208, portal 4 00000100:00000001:1.0:1788572342.650089:0:2384:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788572342.650091:0:2384:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860991090048 : -111212718461568 : ffff9ada456bed80) 00000100:00000040:1.0:1788572342.650097:0:2384:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada456bed80 x1875449616094208/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572358 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572342.650102:0:2384:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788572342.650103:0:2384:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:1.0:1788572342.650106:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2c38. 00000100:00000200:1.0:1788572342.650107:0:2384:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1875449616094208, offset 0 00000400:00000200:1.0:1788572342.650112:0:2384:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:1.0:1788572342.650115:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572342.650118:0:2384:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:1.0:1788572342.650121:0:2384:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788572342.650123:0:2384:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:1.0:1788572342.650124:0:2384:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788572342.650126:0:2384:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:1.0:1788572342.650129:0:2384:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22308:22308:253:4294967295] 192.168.202.113@tcp LPNI seq info [22304:22304:5:4294967295] 00000400:00000200:1.0:1788572342.650133:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572342.650138:0:2384:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:1.0:1788572342.650142:0:2384:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:1.0:1788572342.650145:0:2384:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada48e6fc00. 00000800:00000200:1.0:1788572342.650148:0:2384:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:1.0:1788572342.650152:0:2384:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:1.0:1788572342.650154:0:2384:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada48e6fc00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788572342.650157:0:2384:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572342.650160:0:2384:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: using a timeout of 19833 usecs (20 jiffies) 00000400:00000200:1.0:1788572342.650163:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:1.0:1788572342.650164:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298927639 00000100:00000001:1.0:1788572342.650166:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298938190 00000100:00000001:1.0:1788572342.650168:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298927639 00000100:00000001:1.0:1788572342.650170:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298938190 00000100:00000001:1.0:1788572342.650172:0:2384:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572342.650174:0:2384:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572342.650176:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572342.650178:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572342.650195:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572342.650198:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788572342.650416:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788572342.650421:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada48e6fa00. 00000400:00000200:3.0:1788572342.650427:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572342.650433:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:3.0:1788572342.650440:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d23b8 00000400:00000010:3.0:1788572342.650443:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d23b8. 00000100:00000001:3.0:1788572342.650448:0:2378:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:3.0:1788572342.650453:0:2378:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada456bd880 x1875449616093824/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572358 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572342.650464:0:2378:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572342.650467:0:2378:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada456bd880 x1875449616093824/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572358 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572342.650473:0:2378:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572342.650474:0:2378:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000200:3.0:1788572342.650488:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788572342.650490:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada48e6f400. 00000400:00000200:3.0:1788572342.650492:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572342.650496:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:3.0:1788572342.650498:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2e58 00000400:00000010:3.0:1788572342.650500:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2e58. 00000100:00000001:3.0:1788572342.650503:0:2378:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:3.0:1788572342.650507:0:2378:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada456bd180 x1875449616094080/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572358 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572342.650513:0:2378:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572342.650516:0:2378:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada456bd180 x1875449616094080/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572358 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572342.650522:0:2378:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572342.650524:0:2378:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000200:3.0:1788572342.650532:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788572342.650534:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada48e6f100. 00000400:00000200:3.0:1788572342.650536:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572342.650540:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:3.0:1788572342.650542:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2ee0 00000400:00000010:3.0:1788572342.650544:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2ee0. 00000100:00000001:3.0:1788572342.650547:0:2378:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:3.0:1788572342.650550:0:2378:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada456bfb80 x1875449616093952/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572358 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572342.650556:0:2378:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572342.650559:0:2378:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada456bfb80 x1875449616093952/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572358 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572342.650564:0:2378:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572342.650566:0:2378:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000200:3.0:1788572342.650639:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788572342.650642:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada48e6fc00. 00000400:00000200:3.0:1788572342.650645:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572342.650649:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:3.0:1788572342.650652:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2c38 00000400:00000010:3.0:1788572342.650654:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2c38. 00000100:00000001:3.0:1788572342.650656:0:2378:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:3.0:1788572342.650660:0:2378:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada456bed80 x1875449616094208/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572358 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572342.650666:0:2378:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572342.650670:0:2378:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada456bed80 x1875449616094208/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572358 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572342.650676:0:2378:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572342.650678:0:2378:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000001:3.0:1788572342.660646:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572342.660670:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572342.660675:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572342.660678:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572342.660687:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572342.660697:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572342.660708:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572342.660714:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572342.660720:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572342.660725:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 4 MB=0x6a9b624295400 00000400:00000200:3.0:1788572342.660731:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572342.660737:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.202.113@tcp of length 224/224 into md 0x35825 [1] + 224 00000800:00000001:3.0:1788572342.660745:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572342.660750:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572342.660754:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572342.660759:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572342.660768:0:2377:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572342.660779:0:2377:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada456bed80 x1875449616094208/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572358 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572342.660791:0:2377:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada456bed80 x1875449616094208/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572358 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572342.660830:0:2377:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572342.660842:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572342.660848:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572342.660850:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572342.660898:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572342.660902:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572342.660907:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572342.660912:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572342.660914:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572342.660918:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572342.660920:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 25 MB=0x6a9b624295280 00000400:00000200:3.0:1788572342.660923:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572342.660927:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.202.113@tcp of length 224/224 into md 0x3580d [1] + 224 00000800:00000001:3.0:1788572342.660930:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572342.660934:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572342.660936:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572342.660939:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572342.660942:0:2377:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572342.660946:0:2377:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada456bd880 x1875449616093824/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572358 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572342.660955:0:2377:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada456bd880 x1875449616093824/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572358 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572342.660962:0:2377:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572342.660965:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572342.660968:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572342.660969:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572342.660971:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572342.660974:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572342.660977:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572342.660980:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572342.660983:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572342.660985:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572342.660988:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 10 MB=0x6a9b624295380 00000400:00000200:3.0:1788572342.660991:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572342.660995:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.113@tcp of length 224/224 into md 0x35815 [1] + 224 00000800:00000001:3.0:1788572342.660999:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572342.661002:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572342.661005:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572342.661035:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572342.661039:0:2377:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572342.661043:0:2377:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada456bd180 x1875449616094080/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572358 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572342.661052:0:2377:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada456bd180 x1875449616094080/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572358 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572342.661059:0:2377:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572342.661063:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572342.661067:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572342.661068:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572342.661070:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572342.661073:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572342.661076:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572342.661080:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572342.661082:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572342.661085:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572342.661088:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 10 MB=0x6a9b624295300 00000400:00000200:3.0:1788572342.661091:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572342.661095:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.113@tcp of length 224/224 into md 0x3581d [1] + 224 00000800:00000001:3.0:1788572342.661098:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572342.661104:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572342.661106:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572342.661109:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572342.661112:0:2377:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572342.661116:0:2377:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada456bfb80 x1875449616093952/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572358 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572342.661124:0:2377:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada456bfb80 x1875449616093952/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572358 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572342.661131:0:2377:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572342.661134:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572342.661138:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572342.661833:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572342.661840:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788572342.661850:0:2384:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788572342.661881:0:2384:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788572342.661886:0:2384:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada600d1bb0. 00000400:00000200:1.0:1788572342.661893:0:2384:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d25d8 00000400:00000010:1.0:1788572342.661896:0:2384:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d25d8. 00000100:00000001:1.0:1788572342.661901:0:2384:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:1.0:1788572342.661910:0:2384:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada456bfb80 x1875449616093952/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572358 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788572342.661919:0:2384:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada456bfb80 x1875449616093952/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572358 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572342.661925:0:2384:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788572342.661928:0:2384:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572342.661931:0:2384:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1788572342.661935:0:2384:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788572342.661938:0:2384:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788572342.661941:0:2384:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572342.661943:0:2384:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788572342.661955:0:2384:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada51140460 time=37 v=5 (5 5 5 5) 00000100:00001000:1.0:1788572342.661959:0:2384:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada51140418 time=133 v=5 (2 0 0 0) 00000100:00000001:1.0:1788572342.661962:0:2384:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788572342.661964:0:2384:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572342.661967:0:2384:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788572342.661971:0:2384:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572342.661974:0:2384:0:(client.c:3110:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788572342.661976:0:2384:0:(client.c:3122:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: skip recheck: last_committed 219043334508 00000100:00000001:1.0:1788572342.661978:0:2384:0:(client.c:3123:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788572342.661980:0:2384:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572342.661984:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada456bfb80 x1875449616093952/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572358 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572342.661992:0:2384:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572342.661995:0:2384:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788572342.661997:0:2384:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788572342.662001:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada456bfb80 x1875449616093952/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572358 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572342.662024:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada456bfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616093952:192.168.202.113@tcp:400:kworker.0 00000400:00000001:1.0:1788572342.662030:0:2384:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788572342.662032:0:2384:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788572342.662034:0:2384:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada600d16e0. 00000400:00000200:1.0:1788572342.662037:0:2384:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2088 00000400:00000010:1.0:1788572342.662038:0:2384:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2088. 00000100:00000001:1.0:1788572342.662040:0:2384:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:1.0:1788572342.662044:0:2384:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada456bed80 x1875449616094208/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572358 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788572342.662052:0:2384:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada456bed80 x1875449616094208/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572358 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572342.662058:0:2384:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788572342.662059:0:2384:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572342.662061:0:2384:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1788572342.662062:0:2384:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788572342.662063:0:2384:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788572342.662065:0:2384:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572342.662067:0:2384:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788572342.662071:0:2384:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada48407c60 time=37 v=5 (5 5 5 5) 00000100:00001000:1.0:1788572342.662074:0:2384:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada48407c18 time=7 v=5 (1 0 0 0) 00000100:00000001:1.0:1788572342.662075:0:2384:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788572342.662077:0:2384:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572342.662078:0:2384:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788572342.662080:0:2384:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572342.662082:0:2384:0:(client.c:3110:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788572342.662083:0:2384:0:(client.c:3122:ptlrpc_free_committed()) lustre-OST0000-osc-ffff9ada58b0a000: skip recheck: last_committed 8589935763 00000100:00000001:1.0:1788572342.662085:0:2384:0:(client.c:3123:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788572342.662086:0:2384:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572342.662089:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada456bed80 x1875449616094208/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572358 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572342.662094:0:2384:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572342.662097:0:2384:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788572342.662098:0:2384:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788572342.662101:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada456bed80 x1875449616094208/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572358 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572342.662107:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada456bed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616094208:192.168.202.113@tcp:400:kworker.0 00000400:00000001:1.0:1788572342.662112:0:2384:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788572342.662113:0:2384:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788572342.662115:0:2384:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada600d1c60. 00000400:00000200:1.0:1788572342.662117:0:2384:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2aa0 00000400:00000010:1.0:1788572342.662119:0:2384:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2aa0. 00000100:00000001:1.0:1788572342.662121:0:2384:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:1.0:1788572342.662124:0:2384:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada456bd880 x1875449616093824/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572358 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788572342.662131:0:2384:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada456bd880 x1875449616093824/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572358 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572342.662137:0:2384:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788572342.662138:0:2384:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572342.662140:0:2384:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1788572342.662141:0:2384:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788572342.662142:0:2384:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788572342.662144:0:2384:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572342.662145:0:2384:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788572342.662148:0:2384:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada58a62c60 time=104 v=5 (5 5 5 5) 00000100:00001000:1.0:1788572342.662151:0:2384:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada58a62c18 time=134 v=5 (1 0 0 0) 00000100:00000001:1.0:1788572342.662153:0:2384:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788572342.662154:0:2384:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572342.662155:0:2384:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788572342.662158:0:2384:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572342.662159:0:2384:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572342.662162:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada456bd880 x1875449616093824/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572358 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572342.662168:0:2384:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572342.662170:0:2384:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788572342.662171:0:2384:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788572342.662174:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada456bd880 x1875449616093824/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572358 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572342.662181:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada456bd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449616093824:192.168.202.113@tcp:400:kworker.0 00000400:00000001:1.0:1788572342.662185:0:2384:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788572342.662186:0:2384:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788572342.662188:0:2384:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada600d1ec8. 00000400:00000200:1.0:1788572342.662190:0:2384:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d22a8 00000400:00000010:1.0:1788572342.662191:0:2384:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d22a8. 00000100:00000001:1.0:1788572342.662193:0:2384:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:1.0:1788572342.662196:0:2384:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada456bd180 x1875449616094080/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572358 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788572342.662204:0:2384:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada456bd180 x1875449616094080/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572358 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572342.662210:0:2384:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788572342.662212:0:2384:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572342.662214:0:2384:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1788572342.662215:0:2384:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788572342.662216:0:2384:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788572342.662218:0:2384:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572342.662219:0:2384:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788572342.662223:0:2384:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada58b08460 time=104 v=5 (5 5 5 5) 00000100:00001000:1.0:1788572342.662226:0:2384:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada58b08418 time=104 v=5 (1 1 1 1) 00000100:00000001:1.0:1788572342.662229:0:2384:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788572342.662230:0:2384:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572342.662232:0:2384:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788572342.662234:0:2384:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572342.662236:0:2384:0:(client.c:3110:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788572342.662237:0:2384:0:(client.c:3122:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff9ada58b0a000: skip recheck: last_committed 0 00000100:00000001:1.0:1788572342.662239:0:2384:0:(client.c:3123:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788572342.662241:0:2384:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572342.662245:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada456bd180 x1875449616094080/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572358 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572342.662253:0:2384:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572342.662254:0:2384:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788572342.662268:0:2384:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788572342.662273:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada456bd180 x1875449616094080/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572358 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572342.662281:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada456bd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616094080:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572342.662286:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572342.662289:0:2384:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788572342.662293:0:2384:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada456bfb80 x1875449616093952/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572358 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572342.662300:0:2384:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788572342.662303:0:2384:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788572342.662307:0:2384:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada607e1e00. 02000000:00000001:1.0:1788572342.662312:0:2384:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788572342.662316:0:2384:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788572342.662318:0:2384:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=3 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:1.0:1788572342.662321:0:2384:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1788572342.662324:0:2384:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada48e6f800. 02000000:00000001:1.0:1788572342.662328:0:2384:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788572342.662330:0:2384:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788572342.662332:0:2384:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada456bfb80. 00000100:00000001:1.0:1788572342.662336:0:2384:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788572342.662338:0:2384:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572342.662339:0:2384:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788572342.662343:0:2384:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada456bed80 x1875449616094208/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572358 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572342.662350:0:2384:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788572342.662351:0:2384:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788572342.662353:0:2384:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada607e1800. 02000000:00000001:1.0:1788572342.662355:0:2384:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788572342.662357:0:2384:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788572342.662359:0:2384:0:(genops.c:1223:class_import_put()) import ffff9ada48407800 refcount=2 obd=lustre-OST0000-osc-ffff9ada58b0a000 00000020:00000001:1.0:1788572342.662361:0:2384:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1788572342.662362:0:2384:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada48e6ff00. 02000000:00000001:1.0:1788572342.662364:0:2384:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788572342.662365:0:2384:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788572342.662366:0:2384:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada456bed80. 00000100:00000001:1.0:1788572342.662369:0:2384:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788572342.662370:0:2384:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572342.662371:0:2384:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788572342.662375:0:2384:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada456bd880 x1875449616093824/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572358 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572342.662381:0:2384:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788572342.662382:0:2384:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788572342.662384:0:2384:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada607e1200. 02000000:00000001:1.0:1788572342.662386:0:2384:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788572342.662387:0:2384:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788572342.662389:0:2384:0:(genops.c:1223:class_import_put()) import ffff9ada58a62800 refcount=3 obd=MGC192.168.202.113@tcp 00000020:00000001:1.0:1788572342.662390:0:2384:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1788572342.662392:0:2384:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada48e6f600. 02000000:00000001:1.0:1788572342.662393:0:2384:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788572342.662394:0:2384:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788572342.662396:0:2384:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada456bd880. 00000100:00000001:1.0:1788572342.662398:0:2384:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788572342.662399:0:2384:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572342.662400:0:2384:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788572342.662404:0:2384:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada456bd180 x1875449616094080/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572358 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572342.662410:0:2384:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788572342.662411:0:2384:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788572342.662413:0:2384:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada607e0a00. 02000000:00000001:1.0:1788572342.662415:0:2384:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788572342.662416:0:2384:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788572342.662417:0:2384:0:(genops.c:1223:class_import_put()) import ffff9ada58b08000 refcount=3 obd=lustre-MDT0001-mdc-ffff9ada58b0a000 00000020:00000001:1.0:1788572342.662419:0:2384:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1788572342.662420:0:2384:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada48e6fd00. 02000000:00000001:1.0:1788572342.662422:0:2384:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788572342.662424:0:2384:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788572342.662425:0:2384:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada456bd180. 00000100:00000001:1.0:1788572342.662427:0:2384:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788572342.662429:0:2384:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572342.662431:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572342.662441:0:2384:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788572342.662444:0:2384:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572342.662446:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572342.662449:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572343.159368:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572343.159377:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572343.159383:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572343.159384:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572343.159387:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572343.159389:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788572343.159418:0:15204:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788572343.159421:0:15204:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788572343.159423:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572344.183032:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572344.183041:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572344.183059:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572344.183061:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572344.183064:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572344.183066:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788572344.183179:0:33423:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788572344.183183:0:33423:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788572344.183187:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572345.207125:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572345.207135:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572345.207143:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572345.207144:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572345.207148:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572345.207149:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572345.207256:0:15205:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572345.207260:0:15205:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572345.207263:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572346.234300:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572346.234311:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572346.234319:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572346.234325:0:365:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:2.0:1788572346.234328:0:365:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:2.0:1788572346.234332:0:365:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572346.234335:0:365:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:2.0:1788572346.234338:0:365:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572346.234340:0:365:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572346.234343:0:365:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572346.234347:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572346.234348:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788572346.234392:0:15204:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788572346.234396:0:15204:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788572346.234399:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572347.255227:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572347.255238:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572347.255245:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572347.255246:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572347.255250:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572347.255251:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788572347.255700:0:33423:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788572347.255705:0:33423:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788572347.255708:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788572347.767040:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788572347.767056:0:58077:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788572347.767071:0:58077:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada456bd180. 00000020:00000040:1.0:1788572347.767080:0:58077:0:(genops.c:1210:class_import_get()) import ffff9ada58a62800 refcount=4 obd=MGC192.168.202.113@tcp 00000100:00000001:1.0:1788572347.767086:0:58077:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788572347.767089:0:58077:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788572347.767096:0:58077:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788572347.767107:0:58077:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada48e6fd00. 00000100:00000001:1.0:1788572347.767116:0:58077:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572347.767133:0:58077:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS req@ffff9ada456bd180 x1875449616094336/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788572347.767145:0:58077:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788572347.767162:0:58077:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572347.767170:0:58077:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000948e8606] to pc [ptlrpcd_00_01+1] req@ffff9ada456bd180 x1875449616094336/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572347.767197:0:58077:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572347.767201:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788572347.767205:0:58077:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788572347.767208:0:58077:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada456bd880. 00000020:00000040:1.0:1788572347.767210:0:58077:0:(genops.c:1210:class_import_get()) import ffff9ada51140000 refcount=4 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000100:00000001:1.0:1788572347.767213:0:58077:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788572347.767215:0:58077:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788572347.767216:0:58077:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788572347.767220:0:58077:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada48e6f600. 00000100:00000001:1.0:1788572347.767225:0:58077:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572347.767241:0:58077:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID req@ffff9ada456bd880 x1875449616094464/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788572347.767249:0:58077:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788572347.767251:0:58077:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572347.767254:0:58077:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000082674ba1] to pc [ptlrpcd_00_00+0] req@ffff9ada456bd880 x1875449616094464/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572347.767263:0:58077:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572347.767265:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788572347.767269:0:58077:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788572347.767272:0:58077:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada456bed80. 00000020:00000040:1.0:1788572347.767275:0:58077:0:(genops.c:1210:class_import_get()) import ffff9ada58b08000 refcount=4 obd=lustre-MDT0001-mdc-ffff9ada58b0a000 00000100:00000001:1.0:1788572347.767277:0:58077:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788572347.767278:0:58077:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788572347.767280:0:58077:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788572347.767284:0:58077:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada48e6ff00. 00000100:00000001:1.0:1788572347.767287:0:58077:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572347.767291:0:58077:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID req@ffff9ada456bed80 x1875449616094592/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788572347.767299:0:58077:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788572347.767301:0:58077:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572347.767304:0:58077:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000076fe71b4] to pc [ptlrpcd_00_01+1] req@ffff9ada456bed80 x1875449616094592/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572347.767312:0:58077:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572347.767314:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788572347.767318:0:58077:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788572347.767321:0:58077:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada456bfb80. 00000020:00000040:1.0:1788572347.767323:0:58077:0:(genops.c:1210:class_import_get()) import ffff9ada48407800 refcount=3 obd=lustre-OST0000-osc-ffff9ada58b0a000 00000100:00000001:1.0:1788572347.767326:0:58077:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788572347.767328:0:58077:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788572347.767329:0:58077:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788572347.767332:0:58077:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada48e6f800. 00000100:00000001:1.0:1788572347.767335:0:58077:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572347.767338:0:58077:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0000_UUID req@ffff9ada456bfb80 x1875449616094720/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788572347.767346:0:58077:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788572347.767348:0:58077:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572347.767352:0:58077:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000acbb987d] to pc [ptlrpcd_00_00+0] req@ffff9ada456bfb80 x1875449616094720/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572347.767360:0:58077:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572347.767362:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1788572347.767366:0:58077:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:1.0:1788572347.767372:0:58077:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (4270) 00000100:00000001:1.0:1788572347.767650:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572347.767655:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788572347.767659:0:2384:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788572347.767665:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada456bd180 x1875449616094336/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572347.767673:0:2384:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788572347.767676:0:2384:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572347.767681:0:2384:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788572347.767684:0:2384:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788572347.767687:0:2384:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788572347.767690:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada456bd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449616094336:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572347.767695:0:2384:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788572347.767702:0:2384:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788572347.767705:0:2384:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572347.767708:0:2384:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788572347.767713:0:2384:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada607e0a00. 02000000:00000001:1.0:1788572347.767715:0:2384:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788572347.767741:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:1.0:1788572347.767746:0:2384:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada600d1ec8. 00000400:00000010:1.0:1788572347.767755:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d22a8. 00000100:00000200:1.0:1788572347.767765:0:2384:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616094336, portal 25 00000100:00000001:1.0:1788572347.767768:0:2384:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788572347.767770:0:2384:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860991082880 : -111212718468736 : ffff9ada456bd180) 00000100:00000040:1.0:1788572347.767776:0:2384:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada456bd180 x1875449616094336/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572363 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572347.767785:0:2384:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788572347.767789:0:2384:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:1.0:1788572347.767793:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2aa0. 00000100:00000200:1.0:1788572347.767797:0:2384:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875449616094336, offset 0 00000400:00000200:1.0:1788572347.767805:0:2384:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:1.0:1788572347.767816:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572347.767826:0:2384:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:1.0:1788572347.767832:0:2384:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788572347.767838:0:2384:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:1.0:1788572347.767843:0:2384:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788572347.767847:0:2384:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:1.0:1788572347.767881:0:2384:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22309:22309:256:4294967295] 192.168.202.113@tcp LPNI seq info [22305:22305:8:4294967295] 00000400:00000200:1.0:1788572347.767886:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572347.767898:0:2384:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:1.0:1788572347.767905:0:2384:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:1.0:1788572347.767946:0:2384:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada48e6f900. 00000800:00000200:1.0:1788572347.767955:0:2384:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:1.0:1788572347.767964:0:2384:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:1.0:1788572347.767970:0:2384:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada48e6f900 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788572347.767982:0:2384:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788572347.767987:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:1.0:1788572347.767990:0:2384:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS new deadline estimation for cpu 2 is 4298943309 00000100:00000001:1.0:1788572347.767997:0:2384:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4298938190 00000100:00000001:1.0:1788572347.768002:0:2384:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS new deadline estimation for cpu 3 is 4298943309 00000100:00000001:1.0:1788572347.768006:0:2384:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4298938190 00000100:00000001:1.0:1788572347.768009:0:2384:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572347.768012:0:2384:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572347.768016:0:2384:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788572347.768021:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada456bed80 x1875449616094592/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572347.768027:0:2384:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788572347.768029:0:2384:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572347.768031:0:2384:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788572347.768033:0:2384:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788572347.768035:0:2384:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788572347.768039:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada456bed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616094592:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572347.768043:0:2384:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000800:00000200:3.0:1788572347.768044:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 02000000:00000001:1.0:1788572347.768046:0:2384:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788572347.768049:0:2384:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:3.0:1788572347.768051:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada48e6f900. 02000000:00000001:1.0:1788572347.768052:0:2384:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788572347.768058:0:2384:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada607e1c00. 00000400:00000200:3.0:1788572347.768059:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 02000000:00000001:1.0:1788572347.768062:0:2384:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788572347.768070:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572347.768071:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000010:1.0:1788572347.768075:0:2384:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada600d1c60. 00000400:00000010:1.0:1788572347.768081:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2088. 00000400:00000200:3.0:1788572347.768081:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2aa0 00000100:00000200:1.0:1788572347.768084:0:2384:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616094592, portal 10 00000400:00000010:3.0:1788572347.768086:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2aa0. 00000100:00000001:1.0:1788572347.768088:0:2384:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788572347.768091:0:2384:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860991090048 : -111212718461568 : ffff9ada456bed80) 00000100:00000001:3.0:1788572347.768094:0:2378:0:(events.c:35:request_out_callback()) Process entered 00000100:00000040:1.0:1788572347.768102:0:2384:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada456bed80 x1875449616094592/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572363 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788572347.768103:0:2378:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada456bd180 x1875449616094336/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572363 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572347.768125:0:2384:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788572347.768131:0:2384:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000100:00000001:3.0:1788572347.768132:0:2378:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000400:00000010:1.0:1788572347.768134:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d25d8. 00000100:00000040:3.0:1788572347.768139:0:2378:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada456bd180 x1875449616094336/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572363 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788572347.768140:0:2384:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875449616094592, offset 0 00000100:00000001:3.0:1788572347.768163:0:2378:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788572347.768164:0:2384:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000100:00000001:3.0:1788572347.768167:0:2378:0:(events.c:69:request_out_callback()) Process leaving 00000400:00000200:1.0:1788572347.768172:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572347.768177:0:2384:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:1.0:1788572347.768181:0:2384:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788572347.768184:0:2384:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:1.0:1788572347.768187:0:2384:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788572347.768189:0:2384:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:1.0:1788572347.768193:0:2384:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22310:22310:256:4294967295] 192.168.202.113@tcp LPNI seq info [22306:22306:8:4294967295] 00000400:00000200:1.0:1788572347.768197:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572347.768205:0:2384:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:1.0:1788572347.768210:0:2384:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:1.0:1788572347.768214:0:2384:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada48e6f900. 00000800:00000200:1.0:1788572347.768218:0:2384:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:1.0:1788572347.768223:0:2384:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:1.0:1788572347.768225:0:2384:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada48e6f900 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788572347.768238:0:2384:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572347.768242:0:2384:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: using a timeout of 16636 usecs (17 jiffies) 00000400:00000200:1.0:1788572347.768245:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:1.0:1788572347.768247:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298932754 00000100:00000001:1.0:1788572347.768250:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298943309 00000100:00000001:1.0:1788572347.768252:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298932754 00000100:00000001:1.0:1788572347.768255:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298943309 00000100:00000001:1.0:1788572347.768257:0:2384:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572347.768259:0:2384:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572347.768262:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572347.768265:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572347.768275:0:2384:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788572347.768278:0:2384:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788572347.768280:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572347.768281:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788572347.768282:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788572347.768286:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00000001:1.0:1788572347.768288:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572347.768291:0:2384:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788572347.768292:0:2384:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572347.768293:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572347.768294:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788572347.768296:0:2384:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788572347.768300:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada456bd880 x1875449616094464/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572347.768307:0:2384:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788572347.768309:0:2384:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572347.768311:0:2384:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788572347.768313:0:2384:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788572347.768315:0:2384:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788572347.768318:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada456bd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616094464:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572347.768322:0:2384:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788572347.768325:0:2384:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788572347.768326:0:2384:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572347.768328:0:2384:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788572347.768332:0:2384:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada607e0e00. 02000000:00000001:1.0:1788572347.768334:0:2384:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788572347.768337:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:1.0:1788572347.768340:0:2384:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada600d16e0. 00000400:00000010:1.0:1788572347.768343:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2aa0. 00000100:00000200:1.0:1788572347.768345:0:2384:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616094464, portal 10 00000100:00000001:1.0:1788572347.768347:0:2384:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788572347.768349:0:2384:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860991084672 : -111212718466944 : ffff9ada456bd880) 00000100:00000040:1.0:1788572347.768355:0:2384:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada456bd880 x1875449616094464/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572363 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572347.768361:0:2384:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788572347.768363:0:2384:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:1.0:1788572347.768365:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2c38. 00000100:00000200:1.0:1788572347.768367:0:2384:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875449616094464, offset 0 00000400:00000200:1.0:1788572347.768372:0:2384:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:1.0:1788572347.768375:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572347.768379:0:2384:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:1.0:1788572347.768382:0:2384:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788572347.768384:0:2384:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:1.0:1788572347.768386:0:2384:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788572347.768388:0:2384:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:1.0:1788572347.768391:0:2384:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22311:22311:255:4294967295] 192.168.202.113@tcp LPNI seq info [22307:22307:7:4294967295] 00000400:00000200:1.0:1788572347.768394:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572347.768400:0:2384:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:1.0:1788572347.768403:0:2384:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:1.0:1788572347.768406:0:2384:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada48e6fc00. 00000800:00000200:1.0:1788572347.768410:0:2384:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:1.0:1788572347.768414:0:2384:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:1.0:1788572347.768416:0:2384:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada48e6fc00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788572347.768419:0:2384:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572347.768422:0:2384:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: using a timeout of 41266 usecs (42 jiffies) 00000400:00000200:1.0:1788572347.768425:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:1.0:1788572347.768426:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298932779 00000100:00000001:1.0:1788572347.768429:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298943309 00000100:00000001:1.0:1788572347.768431:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298932779 00000100:00000001:1.0:1788572347.768433:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298943309 00000100:00000001:1.0:1788572347.768435:0:2384:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572347.768437:0:2384:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572347.768440:0:2384:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788572347.768444:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada456bfb80 x1875449616094720/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572347.768450:0:2384:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788572347.768451:0:2384:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572347.768453:0:2384:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788572347.768455:0:2384:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788572347.768457:0:2384:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788572347.768460:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada456bfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616094720:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572347.768463:0:2384:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788572347.768465:0:2384:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788572347.768467:0:2384:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572347.768469:0:2384:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788572347.768472:0:2384:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada607e0800. 02000000:00000001:1.0:1788572347.768474:0:2384:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788572347.768477:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:1.0:1788572347.768480:0:2384:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada600d1bb0. 00000400:00000010:1.0:1788572347.768483:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2ee0. 00000100:00000200:1.0:1788572347.768486:0:2384:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616094720, portal 4 00000100:00000001:1.0:1788572347.768488:0:2384:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788572347.768489:0:2384:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860991093632 : -111212718457984 : ffff9ada456bfb80) 00000100:00000040:1.0:1788572347.768495:0:2384:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada456bfb80 x1875449616094720/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572363 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572347.768508:0:2384:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788572347.768511:0:2384:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:1.0:1788572347.768513:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2e58. 00000100:00000200:1.0:1788572347.768516:0:2384:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1875449616094720, offset 0 00000400:00000200:1.0:1788572347.768520:0:2384:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:1.0:1788572347.768524:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572347.768528:0:2384:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:1.0:1788572347.768531:0:2384:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788572347.768534:0:2384:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:1.0:1788572347.768536:0:2384:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788572347.768538:0:2384:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:1.0:1788572347.768541:0:2384:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22312:22312:254:4294967295] 192.168.202.113@tcp LPNI seq info [22308:22308:6:4294967295] 00000400:00000200:1.0:1788572347.768546:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572347.768552:0:2384:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:1.0:1788572347.768557:0:2384:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:1.0:1788572347.768561:0:2384:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada48e6f100. 00000800:00000200:1.0:1788572347.768564:0:2384:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:1.0:1788572347.768569:0:2384:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:1.0:1788572347.768571:0:2384:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada48e6f100 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788572347.768574:0:2384:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572347.768577:0:2384:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: using a timeout of 19831 usecs (20 jiffies) 00000400:00000200:1.0:1788572347.768580:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:1.0:1788572347.768582:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298932757 00000100:00000001:1.0:1788572347.768584:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298943309 00000100:00000001:1.0:1788572347.768586:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298932757 00000100:00000001:1.0:1788572347.768589:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298943309 00000100:00000001:1.0:1788572347.768591:0:2384:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572347.768593:0:2384:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572347.768596:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572347.768598:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572347.768610:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572347.768613:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788572347.769363:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788572347.769369:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada48e6f900. 00000400:00000200:3.0:1788572347.769374:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572347.769382:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:3.0:1788572347.769387:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d25d8 00000400:00000010:3.0:1788572347.769390:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d25d8. 00000100:00000001:3.0:1788572347.769405:0:2377:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:3.0:1788572347.769412:0:2377:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada456bed80 x1875449616094592/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572363 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572347.769424:0:2377:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572347.769427:0:2377:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada456bed80 x1875449616094592/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572363 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572347.769434:0:2377:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572347.769437:0:2377:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000200:3.0:1788572347.769449:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788572347.769451:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada48e6fc00. 00000400:00000200:3.0:1788572347.769454:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572347.769457:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:3.0:1788572347.769460:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2c38 00000400:00000010:3.0:1788572347.769462:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2c38. 00000100:00000001:3.0:1788572347.769464:0:2377:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:3.0:1788572347.769468:0:2377:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada456bd880 x1875449616094464/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572363 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572347.769475:0:2377:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572347.769478:0:2377:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada456bd880 x1875449616094464/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572363 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572347.769484:0:2377:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572347.769486:0:2377:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000200:3.0:1788572347.769515:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788572347.769517:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada48e6f100. 00000400:00000200:3.0:1788572347.769520:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572347.769523:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:3.0:1788572347.769525:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2e58 00000400:00000010:3.0:1788572347.769527:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2e58. 00000100:00000001:3.0:1788572347.769530:0:2377:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:3.0:1788572347.769533:0:2377:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada456bfb80 x1875449616094720/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572363 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572347.769540:0:2377:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572347.769542:0:2377:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada456bfb80 x1875449616094720/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572363 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572347.769548:0:2377:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572347.769549:0:2377:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000001:3.0:1788572347.770494:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572347.770513:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572347.770519:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572347.770521:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572347.770532:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572347.770538:0:2378:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572347.770548:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572347.770553:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572347.770559:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572347.770564:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 25 MB=0x6a9b624295480 00000400:00000200:3.0:1788572347.770570:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572347.770576:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.202.113@tcp of length 224/224 into md 0x3582d [1] + 224 00000800:00000001:3.0:1788572347.770584:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572347.770590:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572347.770594:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572347.770598:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572347.770605:0:2378:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572347.770613:0:2378:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada456bd180 x1875449616094336/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572363 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572347.770624:0:2378:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada456bd180 x1875449616094336/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572363 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572347.770650:0:2378:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572347.770659:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572347.770662:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572347.771188:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572347.771193:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788572347.771201:0:2384:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788572347.771204:0:2384:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788572347.771208:0:2384:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada600d1ec8. 00000400:00000200:1.0:1788572347.771213:0:2384:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d22a8 00000400:00000010:1.0:1788572347.771215:0:2384:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d22a8. 00000100:00000001:1.0:1788572347.771219:0:2384:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:1.0:1788572347.771226:0:2384:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada456bd180 x1875449616094336/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572363 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788572347.771236:0:2384:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada456bd180 x1875449616094336/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572363 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572347.771242:0:2384:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788572347.771244:0:2384:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572347.771246:0:2384:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1788572347.771250:0:2384:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788572347.771253:0:2384:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788572347.771256:0:2384:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572347.771259:0:2384:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788572347.771267:0:2384:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada58a62c60 time=109 v=5 (5 5 5 5) 00000100:00001000:1.0:1788572347.771272:0:2384:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada58a62c18 time=139 v=5 (1 0 0 0) 00000100:00000001:1.0:1788572347.771275:0:2384:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788572347.771276:0:2384:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572347.771280:0:2384:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788572347.771282:0:2384:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572347.771285:0:2384:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572347.771291:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada456bd180 x1875449616094336/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572363 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572347.771298:0:2384:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572347.771301:0:2384:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788572347.771303:0:2384:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788572347.771307:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada456bd180 x1875449616094336/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572363 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572347.771315:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada456bd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449616094336:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572347.771320:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572347.771323:0:2384:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788572347.771327:0:2384:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada456bd180 x1875449616094336/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572363 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572347.771333:0:2384:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788572347.771336:0:2384:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788572347.771340:0:2384:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada607e0a00. 02000000:00000001:1.0:1788572347.771343:0:2384:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788572347.771346:0:2384:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788572347.771361:0:2384:0:(genops.c:1223:class_import_put()) import ffff9ada58a62800 refcount=3 obd=MGC192.168.202.113@tcp 00000020:00000001:1.0:1788572347.771365:0:2384:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1788572347.771367:0:2384:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada48e6fd00. 02000000:00000001:1.0:1788572347.771371:0:2384:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788572347.771373:0:2384:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788572347.771376:0:2384:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada456bd180. 00000100:00000001:1.0:1788572347.771380:0:2384:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788572347.771382:0:2384:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572347.771385:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572347.771387:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572347.771388:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788572347.771390:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572347.771392:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788572347.777679:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572347.777701:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572347.777706:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572347.777709:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572347.777718:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572347.777728:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572347.777739:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572347.777745:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572347.777751:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572347.777763:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 10 MB=0x6a9b624295580 00000400:00000200:3.0:1788572347.777773:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572347.777779:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.113@tcp of length 224/224 into md 0x35835 [1] + 224 00000800:00000001:3.0:1788572347.777789:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572347.777794:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572347.777798:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572347.777803:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572347.777812:0:2377:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572347.777826:0:2377:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada456bed80 x1875449616094592/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572363 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572347.777839:0:2377:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada456bed80 x1875449616094592/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572363 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572347.778616:0:2377:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572347.778634:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572347.778641:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572347.778644:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572347.778647:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572347.778653:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572347.778659:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572347.778666:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572347.778670:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572347.778673:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572347.778677:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 10 MB=0x6a9b624295500 00000400:00000200:3.0:1788572347.778682:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572347.778686:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.113@tcp of length 224/224 into md 0x3583d [1] + 224 00000800:00000001:3.0:1788572347.778692:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572347.778696:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572347.778699:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572347.778702:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572347.778708:0:2377:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572347.778715:0:2377:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada456bd880 x1875449616094464/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572363 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572347.778725:0:2377:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada456bd880 x1875449616094464/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572363 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572347.778739:0:2377:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572347.778742:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572347.778745:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572347.778747:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572347.778749:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572347.778752:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572347.778755:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572347.778759:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572347.778762:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572347.778764:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572347.778779:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 4 MB=0x6a9b624295600 00000400:00000200:3.0:1788572347.778784:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572347.778789:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.202.113@tcp of length 224/224 into md 0x35845 [1] + 224 00000800:00000001:3.0:1788572347.778793:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572347.778799:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572347.778801:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572347.778804:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572347.778807:0:2377:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572347.778812:0:2377:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada456bfb80 x1875449616094720/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572363 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572347.778820:0:2377:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada456bfb80 x1875449616094720/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572363 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572347.778828:0:2377:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572347.778831:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572347.778834:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572347.779189:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572347.779195:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788572347.779204:0:2384:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788572347.779206:0:2384:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788572347.779210:0:2384:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada600d16e0. 00000400:00000200:1.0:1788572347.779215:0:2384:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2aa0 00000400:00000010:1.0:1788572347.779218:0:2384:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2aa0. 00000100:00000001:1.0:1788572347.779223:0:2384:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:1.0:1788572347.779229:0:2384:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada456bd880 x1875449616094464/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572363 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788572347.779238:0:2384:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada456bd880 x1875449616094464/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572363 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572347.779244:0:2384:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788572347.779246:0:2384:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572347.779248:0:2384:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1788572347.779252:0:2384:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788572347.779254:0:2384:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788572347.779257:0:2384:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572347.779259:0:2384:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788572347.779271:0:2384:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada51140460 time=42 v=5 (5 5 5 5) 00000100:00001000:1.0:1788572347.779276:0:2384:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada51140418 time=138 v=5 (2 0 0 0) 00000100:00000001:1.0:1788572347.779279:0:2384:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788572347.779281:0:2384:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572347.779285:0:2384:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788572347.779289:0:2384:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572347.779292:0:2384:0:(client.c:3110:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788572347.779295:0:2384:0:(client.c:3122:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: skip recheck: last_committed 219043334508 00000100:00000001:1.0:1788572347.779297:0:2384:0:(client.c:3123:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788572347.779299:0:2384:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572347.779305:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada456bd880 x1875449616094464/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572363 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572347.779311:0:2384:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572347.779315:0:2384:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788572347.779317:0:2384:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788572347.779321:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada456bd880 x1875449616094464/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572363 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572347.779329:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada456bd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616094464:192.168.202.113@tcp:400:kworker.0 00000400:00000001:1.0:1788572347.779334:0:2384:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788572347.779336:0:2384:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788572347.779338:0:2384:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada600d1bb0. 00000400:00000200:1.0:1788572347.779340:0:2384:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2ee0 00000400:00000010:1.0:1788572347.779342:0:2384:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2ee0. 00000100:00000001:1.0:1788572347.779344:0:2384:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:1.0:1788572347.779347:0:2384:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada456bfb80 x1875449616094720/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572363 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788572347.779355:0:2384:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada456bfb80 x1875449616094720/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572363 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572347.779362:0:2384:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788572347.779363:0:2384:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572347.779365:0:2384:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1788572347.779367:0:2384:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788572347.779368:0:2384:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788572347.779370:0:2384:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572347.779371:0:2384:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788572347.779375:0:2384:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada48407c60 time=42 v=5 (5 5 5 5) 00000100:00001000:1.0:1788572347.779378:0:2384:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada48407c18 time=12 v=5 (1 0 0 0) 00000100:00000001:1.0:1788572347.779380:0:2384:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788572347.779381:0:2384:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572347.779383:0:2384:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788572347.779385:0:2384:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572347.779387:0:2384:0:(client.c:3110:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788572347.779389:0:2384:0:(client.c:3122:ptlrpc_free_committed()) lustre-OST0000-osc-ffff9ada58b0a000: skip recheck: last_committed 8589935763 00000100:00000001:1.0:1788572347.779390:0:2384:0:(client.c:3123:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788572347.779392:0:2384:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572347.779395:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada456bfb80 x1875449616094720/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572363 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572347.779401:0:2384:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572347.779403:0:2384:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788572347.779404:0:2384:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788572347.779407:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada456bfb80 x1875449616094720/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572363 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572347.779415:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada456bfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616094720:192.168.202.113@tcp:400:kworker.0 00000400:00000001:1.0:1788572347.779420:0:2384:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788572347.779421:0:2384:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788572347.779423:0:2384:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada600d1c60. 00000400:00000200:1.0:1788572347.779426:0:2384:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2088 00000400:00000010:1.0:1788572347.779427:0:2384:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2088. 00000100:00000001:1.0:1788572347.779430:0:2384:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:1.0:1788572347.779434:0:2384:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada456bed80 x1875449616094592/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572363 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788572347.779442:0:2384:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada456bed80 x1875449616094592/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572363 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572347.779448:0:2384:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788572347.779449:0:2384:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572347.779450:0:2384:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1788572347.779452:0:2384:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788572347.779453:0:2384:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788572347.779455:0:2384:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572347.779456:0:2384:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788572347.779460:0:2384:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada58b08460 time=109 v=5 (5 5 5 5) 00000100:00001000:1.0:1788572347.779463:0:2384:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada58b08418 time=109 v=5 (1 1 1 1) 00000100:00000001:1.0:1788572347.779465:0:2384:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788572347.779467:0:2384:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572347.779468:0:2384:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788572347.779470:0:2384:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572347.779473:0:2384:0:(client.c:3110:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788572347.779474:0:2384:0:(client.c:3122:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff9ada58b0a000: skip recheck: last_committed 0 00000100:00000001:1.0:1788572347.779476:0:2384:0:(client.c:3123:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788572347.779477:0:2384:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572347.779480:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada456bed80 x1875449616094592/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572363 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572347.779486:0:2384:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572347.779488:0:2384:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788572347.779490:0:2384:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788572347.779493:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada456bed80 x1875449616094592/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572363 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572347.779499:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada456bed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616094592:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572347.779503:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572347.779506:0:2384:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788572347.779510:0:2384:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada456bd880 x1875449616094464/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572363 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572347.779515:0:2384:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788572347.779518:0:2384:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788572347.779521:0:2384:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada607e0e00. 02000000:00000001:1.0:1788572347.779524:0:2384:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788572347.779528:0:2384:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788572347.779529:0:2384:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=3 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:1.0:1788572347.779531:0:2384:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1788572347.779534:0:2384:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada48e6f600. 02000000:00000001:1.0:1788572347.779537:0:2384:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788572347.779539:0:2384:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788572347.779541:0:2384:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada456bd880. 00000100:00000001:1.0:1788572347.779545:0:2384:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788572347.779547:0:2384:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572347.779548:0:2384:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788572347.779551:0:2384:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada456bfb80 x1875449616094720/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572363 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572347.779556:0:2384:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788572347.779557:0:2384:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788572347.779558:0:2384:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada607e0800. 02000000:00000001:1.0:1788572347.779560:0:2384:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788572347.779561:0:2384:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788572347.779562:0:2384:0:(genops.c:1223:class_import_put()) import ffff9ada48407800 refcount=2 obd=lustre-OST0000-osc-ffff9ada58b0a000 00000020:00000001:1.0:1788572347.779564:0:2384:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1788572347.779565:0:2384:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada48e6f800. 02000000:00000001:1.0:1788572347.779566:0:2384:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788572347.779567:0:2384:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788572347.779569:0:2384:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada456bfb80. 00000100:00000001:1.0:1788572347.779571:0:2384:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788572347.779572:0:2384:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572347.779573:0:2384:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788572347.779576:0:2384:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada456bed80 x1875449616094592/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572363 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572347.779581:0:2384:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788572347.779582:0:2384:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788572347.779584:0:2384:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada607e1c00. 02000000:00000001:1.0:1788572347.779586:0:2384:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788572347.779587:0:2384:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788572347.779588:0:2384:0:(genops.c:1223:class_import_put()) import ffff9ada58b08000 refcount=3 obd=lustre-MDT0001-mdc-ffff9ada58b0a000 00000020:00000001:1.0:1788572347.779589:0:2384:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1788572347.779591:0:2384:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada48e6ff00. 02000000:00000001:1.0:1788572347.779592:0:2384:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788572347.779593:0:2384:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788572347.779594:0:2384:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada456bed80. 00000100:00000001:1.0:1788572347.779596:0:2384:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788572347.779597:0:2384:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572347.779600:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572347.779608:0:2384:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788572347.779610:0:2384:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572347.779612:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572347.779614:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572348.279151:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572348.279176:0:365:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:2.0:1788572348.279179:0:365:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:2.0:1788572348.279183:0:365:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572348.279190:0:365:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:2.0:1788572348.279193:0:365:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572348.279195:0:365:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572348.279197:0:365:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572348.279211:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572348.279212:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572348.279217:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572348.279218:0:365:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:2.0:1788572348.279219:0:365:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:2.0:1788572348.279225:0:365:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:2.0:1788572348.279227:0:365:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:2.0:1788572348.279229:0:365:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572348.279230:0:365:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:2.0:1788572348.279231:0:365:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572348.279233:0:365:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572348.279235:0:365:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572348.279467:0:15205:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572348.279471:0:15205:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572348.279474:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572349.303167:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572349.303176:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572349.303184:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572349.303185:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572349.303200:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572349.303209:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788572349.303241:0:15204:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788572349.303244:0:15204:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788572349.303247:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572350.327091:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572350.327101:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572350.327110:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572350.327111:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572350.327115:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572350.327116:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788572350.328061:0:33423:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788572350.328067:0:33423:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788572350.328073:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572351.351180:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572351.351190:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572351.351200:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572351.351201:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572351.351205:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572351.351205:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572351.353463:0:15205:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572351.353472:0:15205:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572351.353478:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572352.375036:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572352.375045:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572352.375055:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572352.375056:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572352.375060:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572352.375061:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788572352.375095:0:15204:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788572352.375098:0:15204:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788572352.375101:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788572352.887367:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788572352.887378:0:58077:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788572352.887400:0:58077:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada45462680. 00000020:00000040:0.0:1788572352.887408:0:58077:0:(genops.c:1210:class_import_get()) import ffff9ada58a62800 refcount=4 obd=MGC192.168.202.113@tcp 00000100:00000001:0.0:1788572352.887413:0:58077:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788572352.887416:0:58077:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788572352.887429:0:58077:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788572352.887440:0:58077:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada5074a500. 00000100:00000001:0.0:1788572352.887448:0:58077:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572352.887462:0:58077:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS req@ffff9ada45462680 x1875449616094848/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788572352.888052:0:58077:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788572352.888059:0:58077:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572352.888071:0:58077:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000e513b575] to pc [ptlrpcd_00_01+1] req@ffff9ada45462680 x1875449616094848/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572352.888100:0:58077:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572352.888104:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788572352.888108:0:58077:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788572352.888113:0:58077:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada45462a00. 00000020:00000040:0.0:1788572352.888116:0:58077:0:(genops.c:1210:class_import_get()) import ffff9ada51140000 refcount=4 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000100:00000001:0.0:1788572352.888119:0:58077:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788572352.888120:0:58077:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788572352.888122:0:58077:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788572352.888127:0:58077:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada5074a600. 00000100:00000001:0.0:1788572352.888131:0:58077:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572352.888135:0:58077:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID req@ffff9ada45462a00 x1875449616094976/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788572352.888142:0:58077:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788572352.888144:0:58077:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572352.888148:0:58077:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000f7941917] to pc [ptlrpcd_00_00+0] req@ffff9ada45462a00 x1875449616094976/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572352.888156:0:58077:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572352.888158:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788572352.888162:0:58077:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788572352.888165:0:58077:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada45460e00. 00000020:00000040:0.0:1788572352.888167:0:58077:0:(genops.c:1210:class_import_get()) import ffff9ada58b08000 refcount=4 obd=lustre-MDT0001-mdc-ffff9ada58b0a000 00000100:00000001:0.0:1788572352.888169:0:58077:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788572352.888171:0:58077:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788572352.888172:0:58077:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788572352.888175:0:58077:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada5074a300. 00000100:00000001:0.0:1788572352.888178:0:58077:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572352.888181:0:58077:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID req@ffff9ada45460e00 x1875449616095104/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788572352.888187:0:58077:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788572352.888189:0:58077:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572352.888192:0:58077:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ac9b582b] to pc [ptlrpcd_00_01+1] req@ffff9ada45460e00 x1875449616095104/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572352.888199:0:58077:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572352.888201:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788572352.888205:0:58077:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788572352.888208:0:58077:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada45461f80. 00000020:00000040:0.0:1788572352.888210:0:58077:0:(genops.c:1210:class_import_get()) import ffff9ada48407800 refcount=3 obd=lustre-OST0000-osc-ffff9ada58b0a000 00000100:00000001:0.0:1788572352.888212:0:58077:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788572352.888213:0:58077:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788572352.888215:0:58077:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788572352.888217:0:58077:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada5074a200. 00000100:00000001:0.0:1788572352.888220:0:58077:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572352.888224:0:58077:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0000_UUID req@ffff9ada45461f80 x1875449616095232/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788572352.888232:0:58077:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788572352.888233:0:58077:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572352.888237:0:58077:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000723b6ee5] to pc [ptlrpcd_00_00+0] req@ffff9ada45461f80 x1875449616095232/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572352.888249:0:58077:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572352.888251:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:0.0:1788572352.888254:0:58077:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:0.0:1788572352.888259:0:58077:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (4275) 00000100:00000001:1.0:1788572352.890450:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572352.890462:0:2383:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788572352.890465:0:2383:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788572352.890489:0:2383:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada45462a00 x1875449616094976/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572352.890506:0:2383:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788572352.890509:0:2383:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572352.890514:0:2383:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788572352.890518:0:2383:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788572352.890521:0:2383:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788572352.890525:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45462a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616094976:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572352.890529:0:2383:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788572352.890536:0:2383:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788572352.890539:0:2383:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572352.890550:0:2383:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788572352.890558:0:2383:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada607e0800. 02000000:00000001:1.0:1788572352.890560:0:2383:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788572352.890568:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:1.0:1788572352.890573:0:2383:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada600d1c60. 00000400:00000010:1.0:1788572352.890581:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2088. 00000100:00000200:1.0:1788572352.890591:0:2383:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616094976, portal 10 00000100:00000001:1.0:1788572352.890593:0:2383:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788572352.890595:0:2383:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860988615168 : -111212720936448 : ffff9ada45462a00) 00000100:00000040:1.0:1788572352.890603:0:2383:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada45462a00 x1875449616094976/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572368 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572352.890611:0:2383:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788572352.890614:0:2383:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:1.0:1788572352.890617:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2ee0. 00000100:00000200:1.0:1788572352.890621:0:2383:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875449616094976, offset 0 00000400:00000200:1.0:1788572352.890628:0:2383:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:1.0:1788572352.890639:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572352.890648:0:2383:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:1.0:1788572352.890654:0:2383:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788572352.890659:0:2383:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:1.0:1788572352.890663:0:2383:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788572352.890665:0:2383:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:1.0:1788572352.890670:0:2383:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22313:22313:256:4294967295] 192.168.202.113@tcp LPNI seq info [22309:22309:8:4294967295] 00000400:00000200:1.0:1788572352.890674:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572352.890684:0:2383:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:1.0:1788572352.890690:0:2383:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:1.0:1788572352.890696:0:2383:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada48e6ff00. 00000800:00000200:1.0:1788572352.890704:0:2383:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:1.0:1788572352.890712:0:2383:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:1.0:1788572352.890716:0:2383:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada48e6ff00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788572352.890736:0:2383:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572352.890741:0:2383:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: using a timeout of 41263 usecs (42 jiffies) 00000400:00000200:1.0:1788572352.890744:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:1.0:1788572352.890748:0:2383:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298937901 00000100:00000001:1.0:1788572352.890752:0:2383:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298943309 00000100:00000001:1.0:1788572352.890754:0:2383:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298937901 00000100:00000001:1.0:1788572352.890756:0:2383:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298943309 00000100:00000001:1.0:1788572352.890758:0:2383:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572352.890762:0:2383:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572352.890765:0:2383:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788572352.890771:0:2383:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada45461f80 x1875449616095232/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572352.890778:0:2383:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788572352.890780:0:2383:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572352.890783:0:2383:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788572352.890785:0:2383:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788572352.890787:0:2383:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788572352.890789:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45461f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616095232:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572352.890794:0:2383:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788572352.890797:0:2383:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788572352.890799:0:2383:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572352.890800:0:2383:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788572352.890803:0:2383:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada607e0e00. 02000000:00000001:1.0:1788572352.890806:0:2383:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788572352.890809:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:1.0:1788572352.890812:0:2383:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada600d1bb0. 00000400:00000010:1.0:1788572352.890816:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2aa0. 00000100:00000200:1.0:1788572352.890820:0:2383:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616095232, portal 4 00000100:00000001:1.0:1788572352.890822:0:2383:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788572352.890823:0:2383:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860988612480 : -111212720939136 : ffff9ada45461f80) 00000100:00000040:1.0:1788572352.890828:0:2383:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada45461f80 x1875449616095232/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572368 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572352.890834:0:2383:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788572352.890836:0:2383:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:1.0:1788572352.890840:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d22a8. 00000100:00000200:1.0:1788572352.890842:0:2383:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1875449616095232, offset 0 00000400:00000200:1.0:1788572352.890846:0:2383:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:1.0:1788572352.890850:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572352.890884:0:2383:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:1.0:1788572352.890888:0:2383:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788572352.890891:0:2383:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:1.0:1788572352.890893:0:2383:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788572352.890896:0:2383:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000800:00000200:3.0:1788572352.890898:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000400:00000200:1.0:1788572352.890899:0:2383:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22314:22314:255:4294967295] 192.168.202.113@tcp LPNI seq info [22310:22310:7:4294967295] 00000800:00000010:3.0:1788572352.890904:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada48e6ff00. 00000400:00000200:1.0:1788572352.890907:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572352.890916:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788572352.890917:0:2383:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:1.0:1788572352.890924:0:2383:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000400:00000200:3.0:1788572352.890928:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000800:00000010:1.0:1788572352.890932:0:2383:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada48e6f800. 00000400:00000200:3.0:1788572352.890934:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2ee0 00000400:00000010:3.0:1788572352.890938:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2ee0. 00000800:00000200:1.0:1788572352.890939:0:2383:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000100:00000001:3.0:1788572352.890945:0:2378:0:(events.c:35:request_out_callback()) Process entered 00000800:00000200:1.0:1788572352.890946:0:2383:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:1.0:1788572352.890951:0:2383:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada48e6f800 type 1, nob 320 niov 1 nkiov 1 00000100:00000200:3.0:1788572352.890951:0:2378:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada45462a00 x1875449616094976/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572368 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572352.890958:0:2383:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572352.890964:0:2383:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: using a timeout of 19827 usecs (20 jiffies) 00000100:00000001:3.0:1788572352.890966:0:2378:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000400:00000200:1.0:1788572352.890970:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000040:3.0:1788572352.890971:0:2378:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada45462a00 x1875449616094976/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572368 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572352.890974:0:2383:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298937880 00000100:00000001:1.0:1788572352.890980:0:2383:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298943309 00000100:00000001:3.0:1788572352.890981:0:2378:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572352.890984:0:2383:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298937880 00000100:00000001:3.0:1788572352.890985:0:2378:0:(events.c:69:request_out_callback()) Process leaving 00000100:00000001:1.0:1788572352.890989:0:2383:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298943309 00000100:00000001:1.0:1788572352.890992:0:2383:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572352.890994:0:2383:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572352.890997:0:2383:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788572352.891000:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788572352.891001:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000010:3.0:1788572352.891004:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada48e6f800. 00000400:00000200:3.0:1788572352.891007:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572352.891009:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:3.0:1788572352.891011:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d22a8 00000400:00000010:3.0:1788572352.891012:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d22a8. 00000100:00000001:1.0:1788572352.891014:0:2383:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788572352.891014:0:2378:0:(events.c:35:request_out_callback()) Process entered 00000100:00000001:1.0:1788572352.891018:0:2383:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000200:3.0:1788572352.891019:0:2378:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada45461f80 x1875449616095232/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572368 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572352.891023:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572352.891027:0:2383:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788572352.891028:0:2378:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788572352.891031:0:2383:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572352.891032:0:2378:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada45461f80 x1875449616095232/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572368 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572352.891037:0:2383:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00000001:3.0:1788572352.891038:0:2378:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572352.891041:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788572352.891041:0:2378:0:(events.c:69:request_out_callback()) Process leaving 00000100:00000001:1.0:1788572352.891046:0:2383:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788572352.891048:0:2383:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572352.891051:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572352.891053:0:2383:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788572352.891054:0:2383:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788572352.891060:0:2383:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada45462680 x1875449616094848/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572352.891068:0:2383:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788572352.891069:0:2383:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572352.891072:0:2383:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788572352.891073:0:2383:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788572352.891076:0:2383:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788572352.891079:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45462680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449616094848:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572352.891083:0:2383:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788572352.891086:0:2383:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788572352.891087:0:2383:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572352.891089:0:2383:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788572352.891092:0:2383:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada607e0a00. 02000000:00000001:1.0:1788572352.891094:0:2383:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788572352.891098:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:1.0:1788572352.891100:0:2383:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada600d16e0. 00000400:00000010:1.0:1788572352.891104:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d23b8. 00000100:00000200:1.0:1788572352.891107:0:2383:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616094848, portal 25 00000100:00000001:1.0:1788572352.891109:0:2383:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788572352.891110:0:2383:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860988614272 : -111212720937344 : ffff9ada45462680) 00000100:00000040:1.0:1788572352.891115:0:2383:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada45462680 x1875449616094848/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572368 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572352.891121:0:2383:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788572352.891124:0:2383:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:1.0:1788572352.891126:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2d48. 00000100:00000200:1.0:1788572352.891129:0:2383:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875449616094848, offset 0 00000400:00000200:1.0:1788572352.891133:0:2383:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:1.0:1788572352.891137:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572352.891141:0:2383:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:1.0:1788572352.891143:0:2383:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788572352.891145:0:2383:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:1.0:1788572352.891147:0:2383:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788572352.891149:0:2383:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:1.0:1788572352.891152:0:2383:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22315:22315:256:4294967295] 192.168.202.113@tcp LPNI seq info [22311:22311:8:4294967295] 00000400:00000200:1.0:1788572352.891157:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572352.891162:0:2383:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:1.0:1788572352.891167:0:2383:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:1.0:1788572352.891170:0:2383:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada48e6f600. 00000800:00000200:1.0:1788572352.891175:0:2383:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:1.0:1788572352.891179:0:2383:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:1.0:1788572352.891182:0:2383:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada48e6f600 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788572352.891194:0:2383:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788572352.891198:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:1.0:1788572352.891200:0:2383:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS new deadline estimation for cpu 2 is 4298948432 00000100:00000001:1.0:1788572352.891208:0:2383:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4298943309 00000100:00000001:1.0:1788572352.891214:0:2383:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS new deadline estimation for cpu 3 is 4298948432 00000100:00000001:1.0:1788572352.891217:0:2383:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4298943309 00000100:00000001:1.0:1788572352.891221:0:2383:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572352.891223:0:2383:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572352.891226:0:2383:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788572352.891231:0:2383:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada45460e00 x1875449616095104/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572352.891239:0:2383:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788572352.891240:0:2383:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572352.891243:0:2383:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788572352.891245:0:2383:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788572352.891248:0:2383:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788572352.891252:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45460e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616095104:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572352.891255:0:2383:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000800:00000200:3.0:1788572352.891257:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 02000000:00000001:1.0:1788572352.891259:0:2383:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 00000800:00000010:3.0:1788572352.891260:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada48e6f600. 02000000:00000001:1.0:1788572352.891262:0:2383:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788572352.891264:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 02000000:00000001:1.0:1788572352.891265:0:2383:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:3.0:1788572352.891270:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 02000000:00000010:1.0:1788572352.891271:0:2383:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada607e0c00. 00000400:00000200:3.0:1788572352.891273:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2d48 02000000:00000001:1.0:1788572352.891275:0:2383:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788572352.891276:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2d48. 00000100:00000001:3.0:1788572352.891281:0:2377:0:(events.c:35:request_out_callback()) Process entered 00000400:00000200:1.0:1788572352.891283:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000200:3.0:1788572352.891296:0:2377:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada45462680 x1875449616094848/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572368 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000010:1.0:1788572352.891303:0:2383:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada600d1ec8. 00000100:00000001:3.0:1788572352.891310:0:2377:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000400:00000010:1.0:1788572352.891311:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2660. 00000100:00000200:1.0:1788572352.891316:0:2383:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616095104, portal 10 00000100:00000001:1.0:1788572352.891319:0:2383:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000040:3.0:1788572352.891320:0:2377:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada45462680 x1875449616094848/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572368 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572352.891321:0:2383:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860988608000 : -111212720943616 : ffff9ada45460e00) 00000100:00000001:3.0:1788572352.891332:0:2377:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572352.891333:0:2383:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada45460e00 x1875449616095104/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572368 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572352.891336:0:2377:0:(events.c:69:request_out_callback()) Process leaving 00000100:00000001:1.0:1788572352.891344:0:2383:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788572352.891347:0:2383:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:1.0:1788572352.891350:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2000. 00000100:00000200:1.0:1788572352.891353:0:2383:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875449616095104, offset 0 00000400:00000200:1.0:1788572352.891358:0:2383:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:1.0:1788572352.891362:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572352.891366:0:2383:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:1.0:1788572352.891369:0:2383:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788572352.891371:0:2383:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:1.0:1788572352.891373:0:2383:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788572352.891376:0:2383:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:1.0:1788572352.891379:0:2383:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22316:22316:256:4294967295] 192.168.202.113@tcp LPNI seq info [22312:22312:8:4294967295] 00000400:00000200:1.0:1788572352.891382:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572352.891389:0:2383:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:1.0:1788572352.891393:0:2383:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:1.0:1788572352.891396:0:2383:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada48e6fd00. 00000800:00000200:1.0:1788572352.891400:0:2383:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:1.0:1788572352.891404:0:2383:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:1.0:1788572352.891407:0:2383:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada48e6fd00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788572352.891412:0:2383:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572352.891415:0:2383:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: using a timeout of 16635 usecs (17 jiffies) 00000400:00000200:1.0:1788572352.891418:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:1.0:1788572352.891420:0:2383:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298937877 00000100:00000001:1.0:1788572352.891422:0:2383:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298948432 00000100:00000001:1.0:1788572352.891424:0:2383:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298937877 00000100:00000001:1.0:1788572352.891426:0:2383:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298948432 00000100:00000001:1.0:1788572352.891429:0:2383:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572352.891431:0:2383:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572352.891434:0:2383:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572352.891436:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572352.891449:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572352.891452:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788572352.892604:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788572352.892612:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada48e6fd00. 00000400:00000200:3.0:1788572352.892619:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572352.892628:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:3.0:1788572352.892636:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2000 00000400:00000010:3.0:1788572352.892638:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2000. 00000100:00000001:3.0:1788572352.892643:0:2378:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:3.0:1788572352.892650:0:2378:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada45460e00 x1875449616095104/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572368 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572352.892835:0:2378:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572352.892841:0:2378:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada45460e00 x1875449616095104/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572368 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572352.892849:0:2378:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572352.892877:0:2378:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000001:3.0:1788572352.897937:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572352.897962:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572352.897968:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572352.897970:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572352.897981:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572352.897990:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572352.898001:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572352.898006:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572352.898013:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572352.898018:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 4 MB=0x6a9b624295800 00000400:00000200:3.0:1788572352.898024:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572352.898031:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.202.113@tcp of length 224/224 into md 0x35855 [1] + 224 00000800:00000001:3.0:1788572352.898041:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572352.898049:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572352.898054:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572352.898060:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572352.898072:0:2377:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572352.898085:0:2377:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada45461f80 x1875449616095232/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572368 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572352.898097:0:2377:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada45461f80 x1875449616095232/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572368 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572352.898118:0:2377:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572352.898138:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572352.898143:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572352.898397:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572352.898402:0:2383:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788572352.898412:0:2383:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788572352.898414:0:2383:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788572352.898426:0:2383:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada600d1bb0. 00000400:00000200:1.0:1788572352.898433:0:2383:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2aa0 00000400:00000010:1.0:1788572352.898435:0:2383:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2aa0. 00000100:00000001:1.0:1788572352.898440:0:2383:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:1.0:1788572352.898446:0:2383:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada45461f80 x1875449616095232/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572368 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788572352.898456:0:2383:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada45461f80 x1875449616095232/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572368 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572352.898462:0:2383:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788572352.898465:0:2383:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572352.898468:0:2383:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1788572352.898471:0:2383:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788572352.898694:0:2383:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788572352.898697:0:2383:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572352.898700:0:2383:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788572352.898713:0:2383:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada48407c60 time=47 v=5 (5 5 5 5) 00000100:00001000:1.0:1788572352.898718:0:2383:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada48407c18 time=17 v=5 (1 0 0 0) 00000100:00000001:1.0:1788572352.898721:0:2383:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788572352.898723:0:2383:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572352.898726:0:2383:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788572352.898730:0:2383:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572352.898734:0:2383:0:(client.c:3110:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788572352.898736:0:2383:0:(client.c:3122:ptlrpc_free_committed()) lustre-OST0000-osc-ffff9ada58b0a000: skip recheck: last_committed 8589935763 00000100:00000001:1.0:1788572352.898739:0:2383:0:(client.c:3123:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788572352.898741:0:2383:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572352.898747:0:2383:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada45461f80 x1875449616095232/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572368 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572352.898762:0:2383:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572352.898766:0:2383:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788572352.898769:0:2383:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788572352.898773:0:2383:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada45461f80 x1875449616095232/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572368 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572352.898780:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45461f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616095232:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572352.898785:0:2383:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572352.898788:0:2383:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788572352.898791:0:2383:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45461f80 x1875449616095232/t0(0) o400->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572368 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572352.898798:0:2383:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788572352.898800:0:2383:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788572352.898804:0:2383:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada607e0e00. 02000000:00000001:1.0:1788572352.898808:0:2383:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788572352.898811:0:2383:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788572352.898813:0:2383:0:(genops.c:1223:class_import_put()) import ffff9ada48407800 refcount=2 obd=lustre-OST0000-osc-ffff9ada58b0a000 00000020:00000001:1.0:1788572352.898815:0:2383:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1788572352.898818:0:2383:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada5074a200. 02000000:00000001:1.0:1788572352.898823:0:2383:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788572352.898825:0:2383:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788572352.898827:0:2383:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45461f80. 00000100:00000001:1.0:1788572352.898831:0:2383:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788572352.898832:0:2383:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572352.898836:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572352.898838:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572352.898840:0:2383:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788572352.898842:0:2383:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572352.898843:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788572352.913832:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572352.913890:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572352.913896:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572352.913900:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572352.913915:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572352.913924:0:2378:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572352.913936:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572352.913941:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572352.913947:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572352.913953:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 25 MB=0x6a9b624295680 00000400:00000200:3.0:1788572352.913958:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572352.913964:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.202.113@tcp of length 224/224 into md 0x3585d [1] + 224 00000800:00000001:3.0:1788572352.913973:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572352.913981:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572352.913985:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572352.913990:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572352.914000:0:2378:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572352.914011:0:2378:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada45462680 x1875449616094848/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572368 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572352.914022:0:2378:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada45462680 x1875449616094848/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572368 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572352.914050:0:2378:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572352.914060:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572352.914064:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572352.914463:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572352.914468:0:2383:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788572352.914477:0:2383:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788572352.914479:0:2383:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788572352.914484:0:2383:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada600d16e0. 00000400:00000200:1.0:1788572352.914489:0:2383:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d23b8 00000400:00000010:1.0:1788572352.914492:0:2383:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d23b8. 00000100:00000001:1.0:1788572352.914497:0:2383:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:1.0:1788572352.914503:0:2383:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada45462680 x1875449616094848/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572368 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788572352.914512:0:2383:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada45462680 x1875449616094848/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572368 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572352.914519:0:2383:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788572352.914521:0:2383:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572352.914524:0:2383:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1788572352.914527:0:2383:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788572352.914529:0:2383:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788572352.914532:0:2383:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572352.914535:0:2383:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788572352.914544:0:2383:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada58a62c60 time=114 v=5 (5 5 5 5) 00000100:00001000:1.0:1788572352.914549:0:2383:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada58a62c18 time=144 v=5 (1 0 0 0) 00000100:00000001:1.0:1788572352.914552:0:2383:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788572352.914554:0:2383:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572352.914557:0:2383:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788572352.914560:0:2383:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572352.914563:0:2383:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572352.914569:0:2383:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada45462680 x1875449616094848/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572368 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572352.914576:0:2383:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572352.914579:0:2383:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788572352.914581:0:2383:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788572352.914585:0:2383:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada45462680 x1875449616094848/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572368 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572352.914594:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45462680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449616094848:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572352.914600:0:2383:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572352.914603:0:2383:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788572352.914607:0:2383:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45462680 x1875449616094848/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572368 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572352.914614:0:2383:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788572352.914617:0:2383:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788572352.914622:0:2383:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada607e0a00. 02000000:00000001:1.0:1788572352.914625:0:2383:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788572352.914628:0:2383:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788572352.914630:0:2383:0:(genops.c:1223:class_import_put()) import ffff9ada58a62800 refcount=3 obd=MGC192.168.202.113@tcp 00000020:00000001:1.0:1788572352.914632:0:2383:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1788572352.914636:0:2383:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada5074a500. 02000000:00000001:1.0:1788572352.914641:0:2383:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788572352.914644:0:2383:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788572352.914646:0:2383:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45462680. 00000100:00000001:1.0:1788572352.914651:0:2383:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788572352.914652:0:2383:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572352.914655:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572352.914658:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572352.914659:0:2383:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788572352.914660:0:2383:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572352.914662:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788572352.922480:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572352.922501:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572352.922506:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572352.922508:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572352.922518:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572352.922529:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572352.922539:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572352.922545:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572352.922551:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572352.922556:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 10 MB=0x6a9b624295780 00000400:00000200:3.0:1788572352.922562:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572352.922568:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.113@tcp of length 224/224 into md 0x35865 [1] + 224 00000800:00000001:3.0:1788572352.922577:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572352.922581:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572352.922585:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572352.922590:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572352.922612:0:2377:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572352.922625:0:2377:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada45460e00 x1875449616095104/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572368 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572352.922637:0:2377:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada45460e00 x1875449616095104/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572368 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572352.922660:0:2377:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572352.922671:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572352.922675:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572352.922677:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572352.922679:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572352.922682:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572352.922685:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572352.922691:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572352.922693:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572352.922696:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572352.922699:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 10 MB=0x6a9b624295700 00000400:00000200:3.0:1788572352.922702:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572352.922707:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.113@tcp of length 224/224 into md 0x3584d [1] + 224 00000800:00000001:3.0:1788572352.922710:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572352.922717:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572352.922720:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572352.922723:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572352.922726:0:2377:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572352.922731:0:2377:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada45462a00 x1875449616094976/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572368 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572352.922739:0:2377:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada45462a00 x1875449616094976/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572368 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572352.922745:0:2377:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572352.922749:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572352.922752:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572352.922945:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572352.922949:0:2383:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788572352.922958:0:2383:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788572352.922960:0:2383:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788572352.922964:0:2383:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada600d1ec8. 00000400:00000200:1.0:1788572352.922970:0:2383:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2660 00000400:00000010:1.0:1788572352.922973:0:2383:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2660. 00000100:00000001:1.0:1788572352.922977:0:2383:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:1.0:1788572352.922982:0:2383:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada45460e00 x1875449616095104/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572368 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788572352.922989:0:2383:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada45460e00 x1875449616095104/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572368 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572352.922995:0:2383:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788572352.922997:0:2383:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572352.923000:0:2383:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1788572352.923004:0:2383:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788572352.923007:0:2383:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788572352.923010:0:2383:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572352.923013:0:2383:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788572352.923024:0:2383:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada58b08460 time=114 v=5 (5 5 5 5) 00000100:00001000:1.0:1788572352.923029:0:2383:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada58b08418 time=114 v=5 (1 1 1 1) 00000100:00000001:1.0:1788572352.923032:0:2383:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788572352.923034:0:2383:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572352.923039:0:2383:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788572352.923043:0:2383:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572352.923046:0:2383:0:(client.c:3110:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788572352.923049:0:2383:0:(client.c:3122:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff9ada58b0a000: skip recheck: last_committed 0 00000100:00000001:1.0:1788572352.923051:0:2383:0:(client.c:3123:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788572352.923053:0:2383:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572352.923058:0:2383:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada45460e00 x1875449616095104/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572368 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572352.923066:0:2383:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572352.923070:0:2383:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788572352.923072:0:2383:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788572352.923077:0:2383:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada45460e00 x1875449616095104/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572368 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572352.923086:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45460e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616095104:192.168.202.113@tcp:400:kworker.0 00000400:00000001:1.0:1788572352.923093:0:2383:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788572352.923094:0:2383:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788572352.923096:0:2383:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada600d1c60. 00000400:00000200:1.0:1788572352.923098:0:2383:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2088 00000400:00000010:1.0:1788572352.923101:0:2383:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2088. 00000100:00000001:1.0:1788572352.923103:0:2383:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:1.0:1788572352.923107:0:2383:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada45462a00 x1875449616094976/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572368 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788572352.923116:0:2383:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada45462a00 x1875449616094976/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572368 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572352.923123:0:2383:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788572352.923125:0:2383:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572352.923127:0:2383:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1788572352.923128:0:2383:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788572352.923129:0:2383:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788572352.923131:0:2383:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572352.923132:0:2383:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788572352.923136:0:2383:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada51140460 time=47 v=5 (5 5 5 5) 00000100:00001000:1.0:1788572352.923139:0:2383:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada51140418 time=143 v=5 (2 0 0 0) 00000100:00000001:1.0:1788572352.923141:0:2383:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788572352.923142:0:2383:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572352.923143:0:2383:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788572352.923145:0:2383:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572352.923147:0:2383:0:(client.c:3110:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788572352.923149:0:2383:0:(client.c:3122:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: skip recheck: last_committed 219043334508 00000100:00000001:1.0:1788572352.923151:0:2383:0:(client.c:3123:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788572352.923152:0:2383:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572352.923155:0:2383:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada45462a00 x1875449616094976/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572368 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572352.923161:0:2383:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572352.923163:0:2383:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788572352.923165:0:2383:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788572352.923168:0:2383:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada45462a00 x1875449616094976/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572368 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572352.923176:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45462a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616094976:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572352.923180:0:2383:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572352.923184:0:2383:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788572352.923187:0:2383:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45460e00 x1875449616095104/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572368 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572352.923194:0:2383:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788572352.923197:0:2383:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788572352.923202:0:2383:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada607e0c00. 02000000:00000001:1.0:1788572352.923206:0:2383:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788572352.923209:0:2383:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788572352.923211:0:2383:0:(genops.c:1223:class_import_put()) import ffff9ada58b08000 refcount=3 obd=lustre-MDT0001-mdc-ffff9ada58b0a000 00000020:00000001:1.0:1788572352.923214:0:2383:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1788572352.923218:0:2383:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada5074a300. 02000000:00000001:1.0:1788572352.923223:0:2383:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788572352.923225:0:2383:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788572352.923227:0:2383:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45460e00. 00000100:00000001:1.0:1788572352.923231:0:2383:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788572352.923232:0:2383:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572352.923234:0:2383:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788572352.923237:0:2383:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45462a00 x1875449616094976/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572368 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572352.923243:0:2383:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788572352.923244:0:2383:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788572352.923246:0:2383:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada607e0800. 02000000:00000001:1.0:1788572352.923248:0:2383:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788572352.923249:0:2383:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788572352.923250:0:2383:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=3 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:1.0:1788572352.923252:0:2383:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1788572352.923253:0:2383:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada5074a600. 02000000:00000001:1.0:1788572352.923255:0:2383:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788572352.923257:0:2383:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788572352.923258:0:2383:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45462a00. 00000100:00000001:1.0:1788572352.923261:0:2383:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788572352.923262:0:2383:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572352.923264:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572352.923274:0:2383:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788572352.923276:0:2383:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572352.923278:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572352.923281:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572353.400912:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572353.400922:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572353.400929:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572353.400930:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572353.400934:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572353.400936:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788572353.402381:0:33423:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788572353.402388:0:33423:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788572353.402394:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572354.423095:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572354.423104:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572354.423111:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572354.423112:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572354.423115:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572354.423117:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572354.423348:0:15205:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572354.423352:0:15205:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572354.423355:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572355.447028:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572355.447037:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572355.447045:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572355.447049:0:365:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:2.0:1788572355.447060:0:365:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:2.0:1788572355.447065:0:365:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572355.447069:0:365:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:2.0:1788572355.447072:0:365:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572355.447074:0:365:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572355.447077:0:365:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572355.447081:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572355.447083:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788572355.447116:0:15204:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788572355.447119:0:15204:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788572355.447122:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572356.472528:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572356.472538:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572356.472546:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572356.472547:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572356.472551:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572356.472553:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788572356.472883:0:33423:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788572356.472888:0:33423:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788572356.472892:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572357.495067:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572357.495080:0:365:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:2.0:1788572357.495082:0:365:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:2.0:1788572357.495086:0:365:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572357.495093:0:365:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:2.0:1788572357.495095:0:365:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572357.495098:0:365:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572357.495100:0:365:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572357.495105:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572357.495106:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572357.495110:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572357.495111:0:365:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:2.0:1788572357.495113:0:365:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:2.0:1788572357.495119:0:365:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:2.0:1788572357.495122:0:365:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:2.0:1788572357.495123:0:365:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572357.495125:0:365:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:2.0:1788572357.495126:0:365:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572357.495127:0:365:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572357.495130:0:365:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572357.495468:0:15205:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572357.495472:0:15205:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572357.495476:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788572358.007515:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788572358.007529:0:77466:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788572358.007544:0:77466:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada45460a80. 00000020:00000040:0.0:1788572358.007552:0:77466:0:(genops.c:1210:class_import_get()) import ffff9ada58a62800 refcount=4 obd=MGC192.168.202.113@tcp 00000100:00000001:0.0:1788572358.007558:0:77466:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788572358.007562:0:77466:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788572358.007568:0:77466:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788572358.007579:0:77466:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada5074a700. 00000100:00000001:0.0:1788572358.007586:0:77466:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572358.007603:0:77466:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS req@ffff9ada45460a80 x1875449616095360/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788572358.007615:0:77466:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788572358.007620:0:77466:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572358.007626:0:77466:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000007c6786e1] to pc [ptlrpcd_00_01+1] req@ffff9ada45460a80 x1875449616095360/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572358.007648:0:77466:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572358.007651:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788572358.007654:0:77466:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788572358.007657:0:77466:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada45463800. 00000020:00000040:0.0:1788572358.007660:0:77466:0:(genops.c:1210:class_import_get()) import ffff9ada51140000 refcount=4 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000100:00000001:0.0:1788572358.007662:0:77466:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788572358.007663:0:77466:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788572358.007665:0:77466:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788572358.007668:0:77466:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada5074ac00. 00000100:00000001:0.0:1788572358.007671:0:77466:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572358.007686:0:77466:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID req@ffff9ada45463800 x1875449616095488/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788572358.007693:0:77466:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788572358.007695:0:77466:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572358.007699:0:77466:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000bcc5551a] to pc [ptlrpcd_00_00+0] req@ffff9ada45463800 x1875449616095488/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572358.007706:0:77466:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572358.007709:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788572358.007712:0:77466:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788572358.007715:0:77466:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada45461c00. 00000020:00000040:0.0:1788572358.007717:0:77466:0:(genops.c:1210:class_import_get()) import ffff9ada58b08000 refcount=4 obd=lustre-MDT0001-mdc-ffff9ada58b0a000 00000100:00000001:0.0:1788572358.007720:0:77466:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788572358.007721:0:77466:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788572358.007722:0:77466:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788572358.007725:0:77466:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada5074af00. 00000100:00000001:0.0:1788572358.007728:0:77466:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572358.007732:0:77466:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID req@ffff9ada45461c00 x1875449616095616/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788572358.007738:0:77466:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788572358.007740:0:77466:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572358.007744:0:77466:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000038c092fd] to pc [ptlrpcd_00_01+1] req@ffff9ada45461c00 x1875449616095616/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572358.007750:0:77466:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572358.007752:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788572358.007756:0:77466:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:0.0:1788572358.007759:0:77466:0:(import.c:2112:ptlrpc_disconnect_and_idle_import()) Process entered 00000100:00000001:0.0:1788572358.007762:0:77466:0:(import.c:1806:ptlrpc_disconnect_prep_req()) Process entered 00000100:00000010:0.0:1788572358.007765:0:77466:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada45463b80. 00000020:00000040:0.0:1788572358.007767:0:77466:0:(genops.c:1210:class_import_get()) import ffff9ada48407800 refcount=3 obd=lustre-OST0000-osc-ffff9ada58b0a000 00000100:00000001:0.0:1788572358.007769:0:77466:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788572358.007770:0:77466:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788572358.007772:0:77466:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788572358.007775:0:77466:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada5074a900. 00000100:00000001:0.0:1788572358.007778:0:77466:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572358.007781:0:77466:0:(import.c:1844:ptlrpc_disconnect_prep_req()) Process leaving (rc=18446632860988619648 : -111212720931968 : ffff9ada45463b80) 00000100:00080000:0.0:1788572358.007784:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada48407800 lustre-OST0000_UUID: changing import state from FULL to CONNECTING 00000100:00080000:0.0:1788572358.007787:0:77466:0:(import.c:2147:ptlrpc_disconnect_and_idle_import()) lustre-OST0000-osc-ffff9ada58b0a000: disconnect after 23s idle 00000100:00000001:0.0:1788572358.007789:0:77466:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788572358.007791:0:77466:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572358.007795:0:77466:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000000fd8ef99] to pc [ptlrpcd_rcv+-1] req@ffff9ada45463b80 x1875449616095744/t0(0) o9->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572358.007817:0:77466:0:(import.c:2151:ptlrpc_disconnect_and_idle_import()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572358.007820:0:77466:0:(pinger.c:123:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572358.007829:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:0.0:1788572358.007833:0:77466:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:0.0:1788572358.007839:0:77466:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (4280) 00000100:00000001:1.0:1788572358.007985:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572358.007989:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788572358.007992:0:2384:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788572358.007999:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada45460a80 x1875449616095360/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572358.008007:0:2384:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788572358.008009:0:2384:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572358.008014:0:2384:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788572358.008016:0:2384:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788572358.008018:0:2384:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788572358.008021:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45460a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449616095360:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572358.008025:0:2384:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788572358.008032:0:2384:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788572358.008035:0:2384:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572358.008038:0:2384:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788572358.008043:0:2384:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada607e0800. 02000000:00000001:1.0:1788572358.008045:0:2384:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788572358.008055:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:1.0:1788572358.008059:0:2384:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada600d1c60. 00000400:00000010:1.0:1788572358.008068:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2088. 00000100:00000200:1.0:1788572358.008078:0:2384:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616095360, portal 25 00000100:00000001:1.0:1788572358.008080:0:2384:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788572358.008082:0:2384:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860988607104 : -111212720944512 : ffff9ada45460a80) 00000100:00000040:1.0:1788572358.008088:0:2384:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada45460a80 x1875449616095360/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572374 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572358.008102:0:2384:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788572358.008105:0:2384:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:1.0:1788572358.008108:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2660. 00000100:00000200:1.0:1788572358.008112:0:2384:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875449616095360, offset 0 00000400:00000200:1.0:1788572358.008119:0:2384:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:1.0:1788572358.008130:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572358.008140:0:2384:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:1.0:1788572358.008147:0:2384:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788572358.008151:0:2384:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:1.0:1788572358.008156:0:2384:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788572358.008159:0:2384:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:1.0:1788572358.008163:0:2384:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22317:22317:256:4294967295] 192.168.202.113@tcp LPNI seq info [22313:22313:8:4294967295] 00000400:00000200:1.0:1788572358.008168:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572358.008180:0:2384:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:1.0:1788572358.008190:0:2384:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:1.0:1788572358.008196:0:2384:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada48e6f400. 00000800:00000200:1.0:1788572358.008203:0:2384:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:1.0:1788572358.008211:0:2384:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:1.0:1788572358.008216:0:2384:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada48e6f400 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788572358.008234:0:2384:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788572358.008240:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:1.0:1788572358.008243:0:2384:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS new deadline estimation for cpu 2 is 4298953549 00000100:00000001:1.0:1788572358.008250:0:2384:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4298948432 00000100:00000001:1.0:1788572358.008254:0:2384:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS new deadline estimation for cpu 3 is 4298953549 00000100:00000001:1.0:1788572358.008257:0:2384:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4298948432 00000100:00000001:1.0:1788572358.008260:0:2384:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572358.008262:0:2384:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572358.008265:0:2384:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788572358.008269:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada45461c00 x1875449616095616/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572358.008275:0:2384:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788572358.008277:0:2384:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572358.008279:0:2384:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788572358.008281:0:2384:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788572358.008283:0:2384:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788572358.008285:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45461c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616095616:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572358.008289:0:2384:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788572358.008291:0:2384:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788572358.008293:0:2384:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572358.008295:0:2384:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788572358.008298:0:2384:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada607e0c00. 02000000:00000001:1.0:1788572358.008300:0:2384:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788572358.008303:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:1.0:1788572358.008306:0:2384:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada600d1ec8. 00000400:00000010:1.0:1788572358.008310:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d23b8. 00000100:00000200:1.0:1788572358.008313:0:2384:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616095616, portal 10 00000100:00000001:1.0:1788572358.008315:0:2384:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788572358.008316:0:2384:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860988611584 : -111212720940032 : ffff9ada45461c00) 00000100:00000040:1.0:1788572358.008322:0:2384:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada45461c00 x1875449616095616/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572374 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572358.008328:0:2384:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788572358.008331:0:2384:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:1.0:1788572358.008334:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2aa0. 00000100:00000200:1.0:1788572358.008336:0:2384:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875449616095616, offset 0 00000400:00000200:1.0:1788572358.008341:0:2384:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:1.0:1788572358.008345:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572358.008348:0:2384:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:1.0:1788572358.008351:0:2384:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788572358.008353:0:2384:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:1.0:1788572358.008355:0:2384:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788572358.008357:0:2384:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:1.0:1788572358.008360:0:2384:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22318:22318:255:4294967295] 192.168.202.113@tcp LPNI seq info [22314:22314:7:4294967295] 00000400:00000200:1.0:1788572358.008364:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572358.008370:0:2384:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:1.0:1788572358.008373:0:2384:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:1.0:1788572358.008377:0:2384:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada48e6fa00. 00000800:00000200:1.0:1788572358.008381:0:2384:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:1.0:1788572358.008385:0:2384:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:1.0:1788572358.008389:0:2384:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada48e6fa00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788572358.008392:0:2384:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572358.008396:0:2384:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: using a timeout of 16666 usecs (17 jiffies) 00000400:00000200:1.0:1788572358.008399:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:1.0:1788572358.008401:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298942994 00000100:00000001:1.0:1788572358.008404:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298953549 00000100:00000001:1.0:1788572358.008407:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298942994 00000100:00000001:1.0:1788572358.008410:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298953549 00000100:00000001:1.0:1788572358.008411:0:2384:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572358.008413:0:2384:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572358.008416:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572358.008419:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572358.008446:0:2384:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788572358.008450:0:2384:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788572358.008452:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572358.008453:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788572358.008455:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788572358.008459:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00000001:1.0:1788572358.008461:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572358.008463:0:2384:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788572358.008465:0:2384:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572358.008467:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572358.008468:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788572358.008469:0:2384:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000800:00000200:3.0:1788572358.008471:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000040:1.0:1788572358.008474:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada45463800 x1875449616095488/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000800:00000010:3.0:1788572358.008479:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada48e6f400. 00000100:00000001:1.0:1788572358.008485:0:2384:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788572358.008487:0:2384:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788572358.008488:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 02000000:00000001:1.0:1788572358.008490:0:2384:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788572358.008494:0:2384:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788572358.008496:0:2384:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788572358.008501:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00100000:1.0:1788572358.008503:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45463800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616095488:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572358.008508:0:2384:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000200:3.0:1788572358.008509:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2660 02000000:00000001:1.0:1788572358.008511:0:2384:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788572358.008514:0:2384:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788572358.008515:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2660. 02000000:00000001:1.0:1788572358.008516:0:2384:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788572358.008522:0:2384:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada607e0a00. 02000000:00000001:1.0:1788572358.008525:0:2384:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572358.008525:0:2378:0:(events.c:35:request_out_callback()) Process entered 00000400:00000200:1.0:1788572358.008531:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000200:3.0:1788572358.008535:0:2378:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada45460a80 x1875449616095360/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572374 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000010:1.0:1788572358.008536:0:2384:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada600d16e0. 00000400:00000010:1.0:1788572358.008542:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2bb0. 00000100:00000200:1.0:1788572358.008547:0:2384:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616095488, portal 10 00000100:00000001:1.0:1788572358.008551:0:2384:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788572358.008553:0:2384:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860988618752 : -111212720932864 : ffff9ada45463800) 00000100:00000040:1.0:1788572358.008563:0:2384:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada45463800 x1875449616095488/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572374 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572358.008567:0:2378:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1788572358.008573:0:2384:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788572358.008575:0:2378:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada45460a80 x1875449616095360/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572374 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1788572358.008579:0:2384:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:1.0:1788572358.008584:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2220. 00000100:00000200:1.0:1788572358.008588:0:2384:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875449616095488, offset 0 00000100:00000001:3.0:1788572358.008590:0:2378:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572358.008593:0:2378:0:(events.c:69:request_out_callback()) Process leaving 00000400:00000200:1.0:1788572358.008595:0:2384:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:1.0:1788572358.008600:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572358.008605:0:2384:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:1.0:1788572358.008608:0:2384:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788572358.008610:0:2384:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:1.0:1788572358.008613:0:2384:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000800:00000200:3.0:1788572358.008614:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000400:00000200:1.0:1788572358.008616:0:2384:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000800:00000010:3.0:1788572358.008619:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada48e6fa00. 00000400:00000200:1.0:1788572358.008622:0:2384:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22319:22319:255:4294967295] 192.168.202.113@tcp LPNI seq info [22315:22315:7:4294967295] 00000400:00000200:3.0:1788572358.008626:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788572358.008630:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572358.008635:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:1.0:1788572358.008643:0:2384:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:1.0:1788572358.008648:0:2384:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000100:00000001:2.0:1788572358.008701:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788572358.008704:0:2382:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788572358.008706:0:2382:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788572358.008713:0:2382:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada45463b80 x1875449616095744/t0(0) o9->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572358.008721:0:2382:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788572358.008723:0:2382:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572358.008725:0:2382:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788572358.008727:0:2382:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788572358.008729:0:2382:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788572358.008732:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45463b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616095744:192.168.202.113@tcp:9:kworker.0 00000100:00000001:2.0:1788572358.008736:0:2382:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788572358.008739:0:2382:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788572358.008741:0:2382:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572358.008743:0:2382:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788572358.008747:0:2382:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada45575200. 02000000:00000001:2.0:1788572358.008749:0:2382:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788572358.008753:0:2382:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:2.0:1788572358.008756:0:2382:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada43d96cb8. 00000400:00000010:2.0:1788572358.008761:0:2382:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada43c8fe58. 00000100:00000200:2.0:1788572358.008765:0:2382:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616095744, portal 4 00000100:00000001:2.0:1788572358.008767:0:2382:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788572358.008768:0:2382:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860988619648 : -111212720931968 : ffff9ada45463b80) 00000100:00000040:2.0:1788572358.008774:0:2382:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada45463b80 x1875449616095744/t0(0) o9->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572368 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572358.008781:0:2382:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788572358.008784:0:2382:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:2.0:1788572358.008786:0:2382:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada43c8f770. 00000100:00000200:2.0:1788572358.008789:0:2382:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1875449616095744, offset 0 00000400:00000200:2.0:1788572358.008793:0:2382:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000800:00000010:1.0:1788572358.008817:0:2384:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada48e6fa00. 00000800:00000200:1.0:1788572358.008821:0:2384:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:1.0:1788572358.008825:0:2384:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:1.0:1788572358.008828:0:2384:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada48e6fa00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788572358.008831:0:2384:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572358.008834:0:2384:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: using a timeout of 41265 usecs (42 jiffies) 00000400:00000200:1.0:1788572358.008837:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:1.0:1788572358.008839:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298943019 00000100:00000001:1.0:1788572358.008841:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298953549 00000100:00000001:1.0:1788572358.008844:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298943019 00000100:00000001:1.0:1788572358.008846:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298953549 00000100:00000001:1.0:1788572358.008848:0:2384:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572358.008850:0:2384:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572358.008896:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572358.008899:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572358.008910:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572358.008912:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788572358.023654:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2aa0 00000400:00000200:2.0:1788572358.024793:0:2382:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572358.024822:0:2382:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:2.0:1788572358.024835:0:2382:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788572358.024841:0:2382:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:2.0:1788572358.024846:0:2382:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788572358.024849:0:2382:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:2.0:1788572358.024876:0:2382:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22320:22320:254:4294967295] 192.168.202.113@tcp LPNI seq info [22316:22316:6:4294967295] 00000400:00000200:2.0:1788572358.024882:0:2382:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572358.024892:0:2382:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:2.0:1788572358.024900:0:2382:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:2.0:1788572358.024911:0:2382:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada509ed300. 00000800:00000200:2.0:1788572358.024920:0:2382:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:2.0:1788572358.024929:0:2382:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:2.0:1788572358.024934:0:2382:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada509ed300 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788572358.024950:0:2382:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572358.024958:0:2382:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: using a timeout of 19822 usecs (20 jiffies) 00000400:00000200:2.0:1788572358.024963:0:2382:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:2.0:1788572358.024967:0:2382:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298943012 00000100:00000001:2.0:1788572358.024972:0:2382:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298953549 00000100:00000001:2.0:1788572358.024975:0:2382:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298943012 00000100:00000001:2.0:1788572358.024978:0:2382:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298953549 00000100:00000001:2.0:1788572358.024981:0:2382:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788572358.024985:0:2382:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572358.024991:0:2382:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572358.024996:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572358.025008:0:2382:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788572358.025011:0:2382:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=5 : 5 : 5) 00000100:00000001:2.0:1788572358.025014:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788572358.025016:0:2382:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788572358.025018:0:2382:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572358.025020:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788572358.025072:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572358.025090:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788572358.025093:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572358.025096:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572358.025101:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572358.025106:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:2.0:1788572358.025114:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572358.025118:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572358.025122:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572358.025126:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 25 MB=0x6a9b624295880 00000400:00000200:2.0:1788572358.025130:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572358.025137:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.202.113@tcp of length 224/224 into md 0x3586d [1] + 224 00000800:00000001:2.0:1788572358.025145:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572358.025148:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572358.025153:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572358.025157:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:2.0:1788572358.025166:0:2377:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:2.0:1788572358.025177:0:2377:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada45460a80 x1875449616095360/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572374 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788572358.025187:0:2377:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada45460a80 x1875449616095360/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572374 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572358.025212:0:2377:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788572358.025225:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572358.025229:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788572358.025231:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572358.025232:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572358.025235:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572358.025239:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:2.0:1788572358.025244:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:1.0:1788572358.025309:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572358.025313:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788572358.025321:0:2384:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788572358.025324:0:2384:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788572358.025327:0:2384:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada600d1c60. 00000400:00000200:1.0:1788572358.025332:0:2384:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2088 00000400:00000010:1.0:1788572358.025335:0:2384:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2088. 00000100:00000001:1.0:1788572358.025339:0:2384:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:1.0:1788572358.025344:0:2384:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada45460a80 x1875449616095360/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572374 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788572358.025354:0:2384:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada45460a80 x1875449616095360/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572374 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572358.025361:0:2384:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788572358.025363:0:2384:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572358.025365:0:2384:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1788572358.025369:0:2384:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788572358.025372:0:2384:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788572358.025374:0:2384:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572358.025377:0:2384:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788572358.025385:0:2384:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada58a62c60 time=120 v=5 (5 5 5 5) 00000100:00001000:1.0:1788572358.025390:0:2384:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada58a62c18 time=150 v=5 (1 0 0 0) 00000100:00000001:1.0:1788572358.025393:0:2384:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788572358.025394:0:2384:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572358.025398:0:2384:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788572358.025400:0:2384:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572358.025403:0:2384:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572358.025408:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada45460a80 x1875449616095360/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572374 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572358.025416:0:2384:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572358.025419:0:2384:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788572358.025421:0:2384:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788572358.025434:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada45460a80 x1875449616095360/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572374 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572358.025442:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45460a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449616095360:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572358.025448:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572358.025451:0:2384:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788572358.025455:0:2384:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45460a80 x1875449616095360/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572374 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572358.025461:0:2384:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788572358.025464:0:2384:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788572358.025467:0:2384:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada607e0800. 02000000:00000001:1.0:1788572358.025471:0:2384:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788572358.025475:0:2384:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788572358.025476:0:2384:0:(genops.c:1223:class_import_put()) import ffff9ada58a62800 refcount=3 obd=MGC192.168.202.113@tcp 00000020:00000001:1.0:1788572358.025479:0:2384:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1788572358.025483:0:2384:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada5074a700. 02000000:00000001:1.0:1788572358.025487:0:2384:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788572358.025490:0:2384:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788572358.025492:0:2384:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45460a80. 00000100:00000001:1.0:1788572358.025496:0:2384:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788572358.025498:0:2384:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572358.025500:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572358.025502:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572358.025503:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788572358.025505:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572358.025506:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1788572358.025839:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2aa0. 00000100:00000001:3.0:1788572358.025845:0:2378:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:3.0:1788572358.025871:0:2378:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada45461c00 x1875449616095616/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572374 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572358.025883:0:2378:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572358.025886:0:2378:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada45461c00 x1875449616095616/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572374 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572358.025893:0:2378:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572358.025895:0:2378:0:(events.c:69:request_out_callback()) Process leaving 00000400:00000200:2.0:1788572358.035133:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572358.035165:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572358.035175:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 10 MB=0x6a9b624295980 00000400:00000200:2.0:1788572358.035182:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572358.035189:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.113@tcp of length 224/224 into md 0x35875 [1] + 224 00000800:00000001:2.0:1788572358.035202:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572358.035215:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572358.035220:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572358.035225:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000800:00000200:3.0:1788572358.048784:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788572358.048798:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada48e6fa00. 00000400:00000200:3.0:1788572358.048806:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572358.048817:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:3.0:1788572358.048825:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2220 00000400:00000010:3.0:1788572358.048829:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2220. 00000100:00000001:3.0:1788572358.048836:0:2378:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:3.0:1788572358.048847:0:2378:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada45463800 x1875449616095488/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572374 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572358.048879:0:2378:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572358.048883:0:2378:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada45463800 x1875449616095488/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572374 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572358.048897:0:2378:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572358.048899:0:2378:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000200:3.0:1788572358.048970:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788572358.048973:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada509ed300. 00000400:00000200:3.0:1788572358.048976:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572358.048980:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:3.0:1788572358.048983:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada43c8f770 00000400:00000010:3.0:1788572358.048985:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada43c8f770. 00000100:00000001:3.0:1788572358.048988:0:2378:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:3.0:1788572358.048993:0:2378:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada45463b80 x1875449616095744/t0(0) o9->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572368 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572358.049000:0:2378:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572358.049002:0:2378:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada45463b80 x1875449616095744/t0(0) o9->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572368 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572358.049008:0:2378:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572358.049010:0:2378:0:(events.c:69:request_out_callback()) Process leaving 00000100:00000001:2.0:1788572358.052468:0:2377:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:2.0:1788572358.052506:0:2377:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada45461c00 x1875449616095616/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572374 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788572358.052522:0:2377:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada45461c00 x1875449616095616/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572374 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572358.052548:0:2377:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788572358.052569:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572358.052582:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572358.055385:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572358.055396:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788572358.055409:0:2384:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788572358.055412:0:2384:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788572358.055417:0:2384:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada600d1ec8. 00000400:00000200:1.0:1788572358.055439:0:2384:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d23b8 00000400:00000010:1.0:1788572358.055443:0:2384:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d23b8. 00000100:00000001:1.0:1788572358.055450:0:2384:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:1.0:1788572358.055466:0:2384:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada45461c00 x1875449616095616/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572374 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788572358.055488:0:2384:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada45461c00 x1875449616095616/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572374 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572358.055497:0:2384:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788572358.055499:0:2384:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572358.055502:0:2384:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1788572358.055506:0:2384:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788572358.055509:0:2384:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788572358.055511:0:2384:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572358.055514:0:2384:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788572358.055525:0:2384:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada58b08460 time=120 v=5 (5 5 5 5) 00000100:00001000:1.0:1788572358.055530:0:2384:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada58b08418 time=120 v=5 (1 1 1 1) 00000100:00000001:1.0:1788572358.055532:0:2384:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788572358.055534:0:2384:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572358.055538:0:2384:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788572358.055543:0:2384:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572358.055548:0:2384:0:(client.c:3110:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788572358.055550:0:2384:0:(client.c:3122:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff9ada58b0a000: skip recheck: last_committed 0 00000100:00000001:1.0:1788572358.055552:0:2384:0:(client.c:3123:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788572358.055554:0:2384:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572358.055558:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada45461c00 x1875449616095616/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572374 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572358.055566:0:2384:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572358.055569:0:2384:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788572358.055571:0:2384:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788572358.055576:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada45461c00 x1875449616095616/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572374 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572358.055582:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45461c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616095616:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572358.055591:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572358.055594:0:2384:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788572358.055597:0:2384:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45461c00 x1875449616095616/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572374 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572358.055604:0:2384:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788572358.055607:0:2384:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788572358.055611:0:2384:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada607e0c00. 02000000:00000001:1.0:1788572358.055617:0:2384:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788572358.055620:0:2384:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788572358.055622:0:2384:0:(genops.c:1223:class_import_put()) import ffff9ada58b08000 refcount=3 obd=lustre-MDT0001-mdc-ffff9ada58b0a000 00000020:00000001:1.0:1788572358.055625:0:2384:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1788572358.055629:0:2384:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada5074af00. 02000000:00000001:1.0:1788572358.055633:0:2384:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788572358.055635:0:2384:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788572358.055638:0:2384:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45461c00. 00000100:00000001:1.0:1788572358.055642:0:2384:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788572358.055643:0:2384:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572358.055647:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572358.055649:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572358.055650:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788572358.055652:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572358.055654:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788572358.056010:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572358.056027:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572358.056030:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572358.056033:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572358.056043:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572358.056050:0:2378:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572358.056059:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572358.056065:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572358.056071:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572358.056076:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 4 MB=0x6a9b624295a00 00000400:00000200:3.0:1788572358.056081:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572358.056088:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.202.113@tcp of length 224/224 into md 0x35885 [1] + 224 00000800:00000001:3.0:1788572358.056097:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572358.056105:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572358.056110:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572358.056116:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572358.056127:0:2378:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572358.056135:0:2378:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada45463b80 x1875449616095744/t0(0) o9->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572368 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572358.056144:0:2378:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada45463b80 x1875449616095744/t0(0) o9->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572368 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572358.056169:0:2378:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572358.056179:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572358.056182:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572358.057331:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788572358.057339:0:2382:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788572358.057348:0:2382:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788572358.057351:0:2382:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788572358.057356:0:2382:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada43d96cb8. 00000400:00000200:2.0:1788572358.057365:0:2382:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada43c8fe58 00000400:00000010:2.0:1788572358.057368:0:2382:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada43c8fe58. 00000100:00000001:2.0:1788572358.057373:0:2382:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:2.0:1788572358.057384:0:2382:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada45463b80 x1875449616095744/t0(0) o9->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572368 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788572358.057395:0:2382:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada45463b80 x1875449616095744/t0(0) o9->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572368 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572358.057403:0:2382:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788572358.057404:0:2382:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572358.057407:0:2382:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:2.0:1788572358.057411:0:2382:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788572358.057414:0:2382:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788572358.057416:0:2382:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572358.057418:0:2382:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788572358.057437:0:2382:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada48407c60 time=53 v=5 (5 5 5 5) 00000100:00001000:2.0:1788572358.057442:0:2382:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada48407c18 time=23 v=5 (1 0 0 0) 00000100:00000001:2.0:1788572358.057444:0:2382:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788572358.057446:0:2382:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572358.057449:0:2382:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788572358.057452:0:2382:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572358.057457:0:2382:0:(client.c:3110:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788572358.057459:0:2382:0:(client.c:3122:ptlrpc_free_committed()) lustre-OST0000-osc-ffff9ada58b0a000: skip recheck: last_committed 8589935763 00000100:00000001:2.0:1788572358.057461:0:2382:0:(client.c:3123:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788572358.057463:0:2382:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572358.057469:0:2382:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada45463b80 x1875449616095744/t0(0) o9->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572368 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572358.057475:0:2382:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788572358.057488:0:2382:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788572358.057490:0:2382:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00080000:2.0:1788572358.057495:0:2382:0:(import.c:2057:ptlrpc_disconnect_idle_interpret()) @@@ inflight=1, refcount=3: rc = 0 req@ffff9ada45463b80 x1875449616095744/t0(0) o9->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572368 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788572358.057510:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada48407800 lustre-OST0000_UUID: changing import state from CONNECTING to IDLE 00000100:00000040:2.0:1788572358.057515:0:2382:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada45463b80 x1875449616095744/t0(0) o9->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572368 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788572358.057523:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45463b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616095744:192.168.202.113@tcp:9:kworker.0 00000100:00000001:2.0:1788572358.057527:0:2382:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572358.057530:0:2382:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572358.057533:0:2382:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45463b80 x1875449616095744/t0(0) o9->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572368 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572358.057540:0:2382:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572358.057542:0:2382:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572358.057546:0:2382:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada45575200. 02000000:00000001:2.0:1788572358.057550:0:2382:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572358.057553:0:2382:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572358.057554:0:2382:0:(genops.c:1223:class_import_put()) import ffff9ada48407800 refcount=2 obd=lustre-OST0000-osc-ffff9ada58b0a000 00000020:00000001:2.0:1788572358.057556:0:2382:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572358.057558:0:2382:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada5074a900. 02000000:00000001:2.0:1788572358.057562:0:2382:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572358.057564:0:2382:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572358.057566:0:2382:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45463b80. 00000100:00000001:2.0:1788572358.057569:0:2382:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572358.057571:0:2382:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572358.057574:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572358.057583:0:2382:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788572358.057585:0:2382:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572358.057587:0:2382:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788572358.057589:0:2382:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788572358.059601:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572358.059625:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788572358.059630:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572358.059633:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572358.059643:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572358.059653:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:2.0:1788572358.059665:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572358.059671:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572358.059685:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572358.059691:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 10 MB=0x6a9b624295900 00000400:00000200:2.0:1788572358.059696:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572358.059703:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.113@tcp of length 224/224 into md 0x3587d [1] + 224 00000800:00000001:2.0:1788572358.059712:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572358.059800:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572358.059805:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572358.059811:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:2.0:1788572358.059820:0:2377:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:2.0:1788572358.059842:0:2377:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada45463800 x1875449616095488/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572374 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788572358.059910:0:2377:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada45463800 x1875449616095488/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572374 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572358.059932:0:2377:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788572358.060065:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572358.060071:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572358.060779:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572358.060785:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788572358.060795:0:2384:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788572358.060797:0:2384:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788572358.060801:0:2384:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada600d16e0. 00000400:00000200:1.0:1788572358.060808:0:2384:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2bb0 00000400:00000010:1.0:1788572358.060811:0:2384:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2bb0. 00000100:00000001:1.0:1788572358.060816:0:2384:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:1.0:1788572358.060835:0:2384:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada45463800 x1875449616095488/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572374 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788572358.060846:0:2384:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada45463800 x1875449616095488/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572374 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572358.060891:0:2384:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788572358.060895:0:2384:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572358.060898:0:2384:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1788572358.060902:0:2384:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788572358.060905:0:2384:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788572358.060907:0:2384:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572358.060910:0:2384:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788572358.060921:0:2384:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada51140460 time=53 v=5 (5 5 5 5) 00000100:00001000:1.0:1788572358.060926:0:2384:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada51140418 time=149 v=5 (2 0 0 0) 00000100:00000001:1.0:1788572358.060929:0:2384:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788572358.060931:0:2384:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572358.060935:0:2384:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788572358.060947:0:2384:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572358.060952:0:2384:0:(client.c:3110:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788572358.060954:0:2384:0:(client.c:3122:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: skip recheck: last_committed 219043334508 00000100:00000001:1.0:1788572358.060957:0:2384:0:(client.c:3123:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788572358.060960:0:2384:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572358.060964:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada45463800 x1875449616095488/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572374 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572358.060971:0:2384:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572358.060975:0:2384:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788572358.060977:0:2384:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788572358.060980:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada45463800 x1875449616095488/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572374 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572358.060987:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45463800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616095488:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572358.060993:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572358.060996:0:2384:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788572358.060999:0:2384:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45463800 x1875449616095488/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572374 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572358.061005:0:2384:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788572358.061008:0:2384:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788572358.061012:0:2384:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada607e0a00. 02000000:00000001:1.0:1788572358.061016:0:2384:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788572358.061020:0:2384:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788572358.061023:0:2384:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=3 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:1.0:1788572358.061026:0:2384:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1788572358.061029:0:2384:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada5074ac00. 02000000:00000001:1.0:1788572358.061033:0:2384:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788572358.061035:0:2384:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788572358.061037:0:2384:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45463800. 00000100:00000001:1.0:1788572358.061041:0:2384:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788572358.061043:0:2384:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572358.061045:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572358.061054:0:2384:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788572358.061057:0:2384:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572358.061059:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572358.061061:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572358.519012:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572358.519020:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572358.519026:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572358.519027:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572358.519030:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572358.519031:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788572358.519059:0:15204:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788572358.519062:0:15204:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788572358.519065:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572359.543118:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572359.543128:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572359.543135:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572359.543137:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572359.543140:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572359.543141:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788572359.543594:0:33423:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788572359.543599:0:33423:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788572359.543603:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572360.567070:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572360.567080:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572360.567087:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572360.567088:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572360.567091:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572360.567093:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572360.567204:0:15205:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572360.567209:0:15205:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572360.567211:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572361.591820:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572361.591831:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572361.591839:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572361.591841:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572361.591846:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572361.591847:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788572361.591922:0:15204:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788572361.591926:0:15204:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788572361.591930:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572362.616363:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572362.616373:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572362.616381:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572362.616383:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572362.616386:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572362.616388:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788572362.616492:0:33423:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788572362.616496:0:33423:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788572362.616499:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788572363.127340:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788572363.127352:0:58077:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788572363.127367:0:58077:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada45462300. 00000020:00000040:0.0:1788572363.127375:0:58077:0:(genops.c:1210:class_import_get()) import ffff9ada58a62800 refcount=4 obd=MGC192.168.202.113@tcp 00000100:00000001:0.0:1788572363.127380:0:58077:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788572363.127384:0:58077:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788572363.127391:0:58077:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788572363.127404:0:58077:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada5074ac00. 00000100:00000001:0.0:1788572363.127413:0:58077:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572363.127428:0:58077:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS req@ffff9ada45462300 x1875449616095872/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788572363.127441:0:58077:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788572363.127447:0:58077:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572363.127454:0:58077:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000009b0eca3] to pc [ptlrpcd_00_00+0] req@ffff9ada45462300 x1875449616095872/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572363.127489:0:58077:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572363.127493:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788572363.127496:0:58077:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788572363.127500:0:58077:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada45462d80. 00000020:00000040:0.0:1788572363.127502:0:58077:0:(genops.c:1210:class_import_get()) import ffff9ada51140000 refcount=4 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000100:00000001:0.0:1788572363.127505:0:58077:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788572363.127506:0:58077:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788572363.127508:0:58077:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788572363.127511:0:58077:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada5074a900. 00000100:00000001:0.0:1788572363.127515:0:58077:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572363.127519:0:58077:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID req@ffff9ada45462d80 x1875449616096000/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788572363.127527:0:58077:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788572363.127529:0:58077:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572363.127544:0:58077:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000db48fa4b] to pc [ptlrpcd_00_01+1] req@ffff9ada45462d80 x1875449616096000/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572363.127554:0:58077:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572363.127557:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788572363.127561:0:58077:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788572363.127564:0:58077:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada45463480. 00000020:00000040:0.0:1788572363.127568:0:58077:0:(genops.c:1210:class_import_get()) import ffff9ada58b08000 refcount=4 obd=lustre-MDT0001-mdc-ffff9ada58b0a000 00000100:00000001:0.0:1788572363.127570:0:58077:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788572363.127572:0:58077:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788572363.127574:0:58077:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788572363.127577:0:58077:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada5074af00. 00000100:00000001:0.0:1788572363.127580:0:58077:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572363.127584:0:58077:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID req@ffff9ada45463480 x1875449616096128/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788572363.127591:0:58077:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788572363.127593:0:58077:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572363.127597:0:58077:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d2559a6e] to pc [ptlrpcd_00_00+0] req@ffff9ada45463480 x1875449616096128/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572363.127604:0:58077:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572363.127607:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:0.0:1788572363.127610:0:58077:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:0.0:1788572363.127613:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:0.0:1788572363.127616:0:58077:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:0.0:1788572363.127622:0:58077:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (4285) 00000100:00000001:1.0:1788572363.127767:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572363.127772:0:2383:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788572363.127776:0:2383:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788572363.127783:0:2383:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada45462300 x1875449616095872/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572363.127791:0:2383:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788572363.127793:0:2383:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572363.127798:0:2383:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788572363.127801:0:2383:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788572363.127803:0:2383:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788572363.127807:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45462300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449616095872:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572363.127811:0:2383:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788572363.127818:0:2383:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788572363.127820:0:2383:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572363.127823:0:2383:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788572363.127829:0:2383:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada607e0a00. 02000000:00000001:1.0:1788572363.127831:0:2383:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788572363.127839:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:1.0:1788572363.127844:0:2383:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada600d16e0. 00000400:00000010:1.0:1788572363.127886:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2bb0. 00000100:00000200:1.0:1788572363.127896:0:2383:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616095872, portal 25 00000100:00000001:1.0:1788572363.127899:0:2383:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788572363.127902:0:2383:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860988613376 : -111212720938240 : ffff9ada45462300) 00000100:00000040:1.0:1788572363.127908:0:2383:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada45462300 x1875449616095872/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572379 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572363.127915:0:2383:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788572363.127919:0:2383:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:1.0:1788572363.127922:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d23b8. 00000100:00000200:1.0:1788572363.127926:0:2383:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875449616095872, offset 0 00000400:00000200:1.0:1788572363.127934:0:2383:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:1.0:1788572363.127945:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572363.127957:0:2383:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:1.0:1788572363.127964:0:2383:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788572363.127970:0:2383:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:1.0:1788572363.127974:0:2383:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788572363.127977:0:2383:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:1.0:1788572363.127981:0:2383:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22321:22321:256:4294967295] 192.168.202.113@tcp LPNI seq info [22317:22317:8:4294967295] 00000400:00000200:1.0:1788572363.127987:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572363.127998:0:2383:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:1.0:1788572363.128004:0:2383:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:1.0:1788572363.128010:0:2383:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada48e6f400. 00000800:00000200:1.0:1788572363.128017:0:2383:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:1.0:1788572363.128037:0:2383:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:1.0:1788572363.128043:0:2383:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada48e6f400 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788572363.128065:0:2383:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788572363.128070:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:1.0:1788572363.128073:0:2383:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS new deadline estimation for cpu 2 is 4298958669 00000100:00000001:1.0:1788572363.128081:0:2383:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4298953549 00000100:00000001:1.0:1788572363.128087:0:2383:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS new deadline estimation for cpu 3 is 4298958669 00000100:00000001:1.0:1788572363.128092:0:2383:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4298953549 00000100:00000001:1.0:1788572363.128094:0:2383:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572363.128098:0:2383:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572363.128102:0:2383:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788572363.128106:0:2383:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada45463480 x1875449616096128/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572363.128113:0:2383:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788572363.128114:0:2383:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572363.128117:0:2383:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788572363.128118:0:2383:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788572363.128121:0:2383:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788572363.128124:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45463480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616096128:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572363.128127:0:2383:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788572363.128130:0:2383:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788572363.128132:0:2383:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572363.128133:0:2383:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788572363.128137:0:2383:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada607e0c00. 02000000:00000001:1.0:1788572363.128139:0:2383:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788572363.128143:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:1.0:1788572363.128146:0:2383:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada600d1ec8. 00000400:00000010:1.0:1788572363.128150:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2088. 00000100:00000200:1.0:1788572363.128153:0:2383:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616096128, portal 10 00000100:00000001:1.0:1788572363.128155:0:2383:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788572363.128156:0:2383:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860988617856 : -111212720933760 : ffff9ada45463480) 00000100:00000040:1.0:1788572363.128164:0:2383:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada45463480 x1875449616096128/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572379 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572363.128170:0:2383:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788572363.128173:0:2383:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:1.0:1788572363.128175:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2b28. 00000100:00000200:1.0:1788572363.128177:0:2383:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875449616096128, offset 0 00000400:00000200:1.0:1788572363.128181:0:2383:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:1.0:1788572363.128185:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572363.128189:0:2383:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:1.0:1788572363.128192:0:2383:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788572363.128195:0:2383:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:1.0:1788572363.128197:0:2383:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788572363.128200:0:2383:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:1.0:1788572363.128203:0:2383:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22322:22322:255:4294967295] 192.168.202.113@tcp LPNI seq info [22318:22318:7:4294967295] 00000400:00000200:1.0:1788572363.128207:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572363.128213:0:2383:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:1.0:1788572363.128217:0:2383:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:1.0:1788572363.128221:0:2383:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada48e6fd00. 00000800:00000200:1.0:1788572363.128236:0:2383:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:1.0:1788572363.128241:0:2383:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:1.0:1788572363.128243:0:2383:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada48e6fd00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788572363.128246:0:2383:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572363.128250:0:2383:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: using a timeout of 16715 usecs (17 jiffies) 00000400:00000200:1.0:1788572363.128253:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:1.0:1788572363.128255:0:2383:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298948114 00000100:00000001:1.0:1788572363.128258:0:2383:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298958669 00000100:00000001:1.0:1788572363.128260:0:2383:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298948114 00000100:00000001:1.0:1788572363.128262:0:2383:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298958669 00000100:00000001:1.0:1788572363.128264:0:2383:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572363.128266:0:2383:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572363.128268:0:2383:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572363.128271:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572363.128286:0:2383:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788572363.128289:0:2383:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788572363.128291:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572363.128292:0:2383:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788572363.128294:0:2383:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788572363.128297:0:2383:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00000001:1.0:1788572363.128298:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572363.128300:0:2383:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788572363.128302:0:2383:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572363.128303:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572363.128304:0:2383:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788572363.128305:0:2383:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788572363.128310:0:2383:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada45462d80 x1875449616096000/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572363.128317:0:2383:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788572363.128318:0:2383:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572363.128320:0:2383:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788572363.128322:0:2383:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788572363.128324:0:2383:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788572363.128326:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45462d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616096000:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572363.128330:0:2383:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788572363.128332:0:2383:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788572363.128334:0:2383:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572363.128335:0:2383:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788572363.128339:0:2383:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada607e0800. 02000000:00000001:1.0:1788572363.128341:0:2383:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788572363.128344:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:1.0:1788572363.128347:0:2383:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada600d1c60. 00000400:00000010:1.0:1788572363.128351:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2908. 00000100:00000200:1.0:1788572363.128354:0:2383:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616096000, portal 10 00000100:00000001:1.0:1788572363.128356:0:2383:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788572363.128357:0:2383:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860988616064 : -111212720935552 : ffff9ada45462d80) 00000100:00000040:1.0:1788572363.128363:0:2383:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada45462d80 x1875449616096000/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572379 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572363.128369:0:2383:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788572363.128372:0:2383:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:1.0:1788572363.128375:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2550. 00000100:00000200:1.0:1788572363.128377:0:2383:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875449616096000, offset 0 00000400:00000200:1.0:1788572363.128381:0:2383:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:1.0:1788572363.128385:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572363.128388:0:2383:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:1.0:1788572363.128391:0:2383:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788572363.128394:0:2383:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:1.0:1788572363.128396:0:2383:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788572363.128399:0:2383:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:1.0:1788572363.128402:0:2383:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22323:22323:254:4294967295] 192.168.202.113@tcp LPNI seq info [22319:22319:6:4294967295] 00000400:00000200:1.0:1788572363.128406:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572363.128412:0:2383:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:1.0:1788572363.128417:0:2383:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:1.0:1788572363.128420:0:2383:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada48e6f600. 00000800:00000200:1.0:1788572363.128424:0:2383:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:1.0:1788572363.128428:0:2383:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:1.0:1788572363.128431:0:2383:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada48e6f600 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788572363.128434:0:2383:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572363.128436:0:2383:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: using a timeout of 41266 usecs (42 jiffies) 00000400:00000200:1.0:1788572363.128440:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:1.0:1788572363.128441:0:2383:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298948139 00000100:00000001:1.0:1788572363.128444:0:2383:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298958669 00000100:00000001:1.0:1788572363.128446:0:2383:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298948139 00000100:00000001:1.0:1788572363.128448:0:2383:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298958669 00000100:00000001:1.0:1788572363.128450:0:2383:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572363.128452:0:2383:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572363.128455:0:2383:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572363.128457:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572363.128474:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572363.128476:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788572363.128727:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788572363.128733:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada48e6f400. 00000400:00000200:2.0:1788572363.128744:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572363.128751:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:2.0:1788572363.128759:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d23b8 00000400:00000010:2.0:1788572363.128762:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d23b8. 00000100:00000001:2.0:1788572363.128770:0:2378:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:2.0:1788572363.128775:0:2378:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada45462300 x1875449616095872/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572379 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572363.128787:0:2378:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572363.128790:0:2378:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada45462300 x1875449616095872/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572379 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572363.128797:0:2378:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572363.128799:0:2378:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000200:2.0:1788572363.128812:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788572363.128815:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada48e6fd00. 00000400:00000200:2.0:1788572363.128817:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572363.128821:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:2.0:1788572363.128823:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2b28 00000400:00000010:2.0:1788572363.128825:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2b28. 00000100:00000001:2.0:1788572363.128828:0:2378:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:2.0:1788572363.128832:0:2378:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada45463480 x1875449616096128/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572379 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572363.128839:0:2378:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572363.128842:0:2378:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada45463480 x1875449616096128/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572379 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572363.128848:0:2378:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572363.128850:0:2378:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000200:2.0:1788572363.128973:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788572363.128977:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada48e6f600. 00000400:00000200:2.0:1788572363.128981:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572363.128986:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:2.0:1788572363.128990:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2550 00000400:00000010:2.0:1788572363.128993:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2550. 00000100:00000001:2.0:1788572363.128997:0:2378:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:2.0:1788572363.129001:0:2378:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada45462d80 x1875449616096000/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572379 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572363.129010:0:2378:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572363.129015:0:2378:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada45462d80 x1875449616096000/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572379 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572363.129041:0:2378:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572363.129046:0:2378:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000001:2.0:1788572363.131512:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572363.131546:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788572363.131552:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572363.131556:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572363.131567:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572363.131577:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:2.0:1788572363.131589:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572363.131594:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572363.131601:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572363.131606:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 25 MB=0x6a9b624295a80 00000400:00000200:2.0:1788572363.131611:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572363.131617:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.202.113@tcp of length 224/224 into md 0x35895 [1] + 224 00000800:00000001:2.0:1788572363.131627:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572363.131631:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572363.131636:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572363.131641:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:2.0:1788572363.131652:0:2377:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:2.0:1788572363.131662:0:2377:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada45462300 x1875449616095872/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572379 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788572363.131675:0:2377:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada45462300 x1875449616095872/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572379 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572363.131706:0:2377:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788572363.131716:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572363.131720:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788572363.131722:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572363.131724:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572363.131727:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572363.131730:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:2.0:1788572363.131735:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572363.131737:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572363.131740:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572363.131743:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 10 MB=0x6a9b624295b80 00000400:00000200:2.0:1788572363.131747:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572363.131753:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.113@tcp of length 224/224 into md 0x3589d [1] + 224 00000800:00000001:2.0:1788572363.131757:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788572363.131766:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572363.131771:0:2383:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000800:00000001:2.0:1788572363.131774:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572363.131777:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000001:1.0:1788572363.131781:0:2383:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:2.0:1788572363.131783:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000001:1.0:1788572363.131785:0:2383:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788572363.131790:0:2383:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada600d16e0. 00000400:00000200:1.0:1788572363.131795:0:2383:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2bb0 00000400:00000010:1.0:1788572363.131798:0:2383:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2bb0. 00000100:00000001:1.0:1788572363.131803:0:2383:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000001:2.0:1788572363.131804:0:2377:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:1.0:1788572363.131810:0:2383:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada45462300 x1875449616095872/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572379 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788572363.131812:0:2377:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada45463480 x1875449616096128/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572379 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788572363.131836:0:2377:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada45463480 x1875449616096128/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572379 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788572363.131841:0:2383:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada45462300 x1875449616095872/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572379 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572363.131848:0:2383:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788572363.131851:0:2383:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572363.131884:0:2383:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1788572363.131888:0:2383:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788572363.131891:0:2383:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788572363.131893:0:2383:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572363.131897:0:2383:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788572363.131906:0:2383:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada58a62c60 time=125 v=5 (5 5 5 5) 00000100:00001000:1.0:1788572363.131910:0:2383:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada58a62c18 time=5 v=5 (1 1 0 0) 00000100:00000001:1.0:1788572363.131912:0:2383:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788572363.131914:0:2383:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572363.131917:0:2383:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788572363.131919:0:2383:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572363.131922:0:2383:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572363.131927:0:2383:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada45462300 x1875449616095872/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572379 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572363.131934:0:2383:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572363.131938:0:2383:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788572363.131940:0:2383:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788572363.131945:0:2383:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada45462300 x1875449616095872/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572379 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572363.131953:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45462300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449616095872:192.168.202.113@tcp:400:kworker.0 00000100:00000001:2.0:1788572363.132012:0:2377:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788572363.132017:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572363.132022:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788572363.132023:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572363.132026:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572363.132030:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572363.132033:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:2.0:1788572363.132038:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572363.132040:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572363.132043:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572363.132046:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 10 MB=0x6a9b624295b00 00000400:00000200:2.0:1788572363.132050:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572363.132054:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.113@tcp of length 224/224 into md 0x358a5 [1] + 224 00000800:00000001:2.0:1788572363.132058:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572363.132066:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572363.132068:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572363.132071:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:2.0:1788572363.132075:0:2377:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:2.0:1788572363.132079:0:2377:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada45462d80 x1875449616096000/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572379 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788572363.132087:0:2377:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada45462d80 x1875449616096000/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572379 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572363.132093:0:2377:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788572363.132096:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572363.132099:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:1.0:1788572363.136566:0:2383:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788572363.136576:0:2383:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788572363.136581:0:2383:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada600d1ec8. 00000400:00000200:1.0:1788572363.136593:0:2383:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2088 00000400:00000010:1.0:1788572363.136596:0:2383:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2088. 00000100:00000001:1.0:1788572363.136604:0:2383:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:1.0:1788572363.136620:0:2383:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada45463480 x1875449616096128/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572379 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788572363.136635:0:2383:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada45463480 x1875449616096128/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572379 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572363.136644:0:2383:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788572363.136647:0:2383:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572363.136650:0:2383:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1788572363.136654:0:2383:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788572363.136657:0:2383:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788572363.136659:0:2383:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572363.136662:0:2383:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788572363.136674:0:2383:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada58b08460 time=125 v=5 (5 5 5 5) 00000100:00001000:1.0:1788572363.136679:0:2383:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada58b08418 time=125 v=5 (1 1 1 1) 00000100:00000001:1.0:1788572363.136682:0:2383:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788572363.136683:0:2383:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572363.136687:0:2383:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788572363.136691:0:2383:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572363.136695:0:2383:0:(client.c:3110:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788572363.136697:0:2383:0:(client.c:3122:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff9ada58b0a000: skip recheck: last_committed 0 00000100:00000001:1.0:1788572363.136699:0:2383:0:(client.c:3123:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788572363.136701:0:2383:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572363.136706:0:2383:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada45463480 x1875449616096128/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572379 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572363.136713:0:2383:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572363.136717:0:2383:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788572363.136719:0:2383:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788572363.136722:0:2383:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada45463480 x1875449616096128/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572379 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572363.136730:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45463480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616096128:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572363.136735:0:2383:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572363.136739:0:2383:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788572363.136744:0:2383:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45462300 x1875449616095872/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572379 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572363.136755:0:2383:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788572363.136758:0:2383:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788572363.136762:0:2383:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada607e0a00. 02000000:00000001:1.0:1788572363.136766:0:2383:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788572363.136769:0:2383:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788572363.136771:0:2383:0:(genops.c:1223:class_import_put()) import ffff9ada58a62800 refcount=3 obd=MGC192.168.202.113@tcp 00000020:00000001:1.0:1788572363.136774:0:2383:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1788572363.136776:0:2383:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada5074ac00. 02000000:00000001:1.0:1788572363.136782:0:2383:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788572363.136784:0:2383:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788572363.136786:0:2383:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45462300. 00000100:00000001:1.0:1788572363.136790:0:2383:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788572363.136791:0:2383:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572363.136793:0:2383:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788572363.136796:0:2383:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45463480 x1875449616096128/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572379 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572363.136801:0:2383:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788572363.136803:0:2383:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788572363.136804:0:2383:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada607e0c00. 02000000:00000001:1.0:1788572363.136806:0:2383:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788572363.136807:0:2383:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788572363.136808:0:2383:0:(genops.c:1223:class_import_put()) import ffff9ada58b08000 refcount=3 obd=lustre-MDT0001-mdc-ffff9ada58b0a000 00000020:00000001:1.0:1788572363.136810:0:2383:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1788572363.136811:0:2383:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada5074af00. 02000000:00000001:1.0:1788572363.136814:0:2383:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788572363.136815:0:2383:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788572363.136816:0:2383:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45463480. 00000100:00000001:1.0:1788572363.136819:0:2383:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788572363.136820:0:2383:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572363.136824:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572363.136827:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572363.136830:0:2383:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788572363.136834:0:2383:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788572363.136836:0:2383:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788572363.136838:0:2383:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada600d1c60. 00000400:00000200:1.0:1788572363.136840:0:2383:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2908 00000400:00000010:1.0:1788572363.136842:0:2383:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2908. 00000100:00000001:1.0:1788572363.136845:0:2383:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:1.0:1788572363.136849:0:2383:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada45462d80 x1875449616096000/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572379 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788572363.136875:0:2383:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada45462d80 x1875449616096000/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572379 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572363.136882:0:2383:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788572363.136884:0:2383:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572363.136886:0:2383:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1788572363.136887:0:2383:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788572363.136888:0:2383:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788572363.136890:0:2383:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572363.136892:0:2383:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788572363.136896:0:2383:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada51140460 time=58 v=5 (5 5 5 5) 00000100:00001000:1.0:1788572363.136899:0:2383:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada51140418 time=154 v=5 (2 0 0 0) 00000100:00000001:1.0:1788572363.136901:0:2383:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788572363.136903:0:2383:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572363.136904:0:2383:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788572363.136906:0:2383:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572363.136908:0:2383:0:(client.c:3110:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788572363.136910:0:2383:0:(client.c:3122:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: skip recheck: last_committed 219043334508 00000100:00000001:1.0:1788572363.136911:0:2383:0:(client.c:3123:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788572363.136913:0:2383:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572363.136916:0:2383:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada45462d80 x1875449616096000/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572379 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572363.136922:0:2383:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572363.136924:0:2383:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788572363.136925:0:2383:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788572363.136928:0:2383:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada45462d80 x1875449616096000/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572379 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572363.136935:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45462d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616096000:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572363.136939:0:2383:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572363.136942:0:2383:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788572363.136945:0:2383:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45462d80 x1875449616096000/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572379 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572363.136951:0:2383:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788572363.136952:0:2383:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788572363.136954:0:2383:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada607e0800. 02000000:00000001:1.0:1788572363.136957:0:2383:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788572363.136958:0:2383:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788572363.136959:0:2383:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=3 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:1.0:1788572363.136961:0:2383:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1788572363.136962:0:2383:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada5074a900. 02000000:00000001:1.0:1788572363.136964:0:2383:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788572363.136965:0:2383:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788572363.136966:0:2383:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45462d80. 00000100:00000001:1.0:1788572363.136968:0:2383:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788572363.136970:0:2383:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572363.136972:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572363.136982:0:2383:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788572363.136984:0:2383:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572363.136986:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572363.136987:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572363.639074:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572363.639084:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572363.639092:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572363.639093:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572363.639097:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572363.639098:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572363.639269:0:15205:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572363.639273:0:15205:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572363.639276:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572364.668620:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572364.668630:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572364.668638:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572364.668639:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572364.668643:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572364.668645:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572364.668916:0:15204:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572364.668920:0:15204:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572364.668923:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572365.687495:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572365.687505:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572365.687513:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572365.687517:0:365:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:2.0:1788572365.687519:0:365:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:2.0:1788572365.687523:0:365:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572365.687527:0:365:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:2.0:1788572365.687529:0:365:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572365.687531:0:365:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572365.687533:0:365:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572365.687537:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572365.687538:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788572365.687772:0:33423:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788572365.687776:0:33423:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788572365.687779:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572366.712830:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572366.712840:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572366.712848:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572366.712849:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572366.712885:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572366.712886:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572366.712987:0:15205:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572366.712991:0:15205:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572366.712994:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572367.738893:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572367.738906:0:365:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:2.0:1788572367.738909:0:365:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:2.0:1788572367.738913:0:365:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572367.738920:0:365:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:2.0:1788572367.738922:0:365:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572367.738924:0:365:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572367.738928:0:365:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572367.738933:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572367.738934:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572367.738938:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572367.738939:0:365:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:2.0:1788572367.738940:0:365:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:2.0:1788572367.738947:0:365:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:2.0:1788572367.738950:0:365:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:2.0:1788572367.738951:0:365:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572367.738953:0:365:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:2.0:1788572367.738954:0:365:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572367.738955:0:365:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572367.738957:0:365:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572367.739074:0:15204:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572367.739078:0:15204:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572367.739081:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788572368.249019:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788572368.249037:0:58077:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788572368.249051:0:58077:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada456bed80. 00000020:00000040:1.0:1788572368.249059:0:58077:0:(genops.c:1210:class_import_get()) import ffff9ada58a62800 refcount=4 obd=MGC192.168.202.113@tcp 00000100:00000001:1.0:1788572368.249064:0:58077:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788572368.249068:0:58077:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788572368.249074:0:58077:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788572368.249085:0:58077:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada48e6f800. 00000100:00000001:1.0:1788572368.249095:0:58077:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572368.249110:0:58077:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS req@ffff9ada456bed80 x1875449616096256/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788572368.249123:0:58077:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788572368.249128:0:58077:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572368.249136:0:58077:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000076fe71b4] to pc [ptlrpcd_00_01+1] req@ffff9ada456bed80 x1875449616096256/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572368.249178:0:58077:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572368.249182:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788572368.249186:0:58077:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788572368.249189:0:58077:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada456bfb80. 00000020:00000040:1.0:1788572368.249191:0:58077:0:(genops.c:1210:class_import_get()) import ffff9ada51140000 refcount=4 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000100:00000001:1.0:1788572368.249193:0:58077:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788572368.249195:0:58077:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788572368.249196:0:58077:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788572368.249199:0:58077:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada48e6ff00. 00000100:00000001:1.0:1788572368.249202:0:58077:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572368.249207:0:58077:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID req@ffff9ada456bfb80 x1875449616096384/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788572368.249214:0:58077:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788572368.249216:0:58077:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572368.249220:0:58077:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000acbb987d] to pc [ptlrpcd_00_00+0] req@ffff9ada456bfb80 x1875449616096384/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572368.249228:0:58077:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572368.249230:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788572368.249233:0:58077:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788572368.249235:0:58077:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada456bd880. 00000020:00000040:1.0:1788572368.249237:0:58077:0:(genops.c:1210:class_import_get()) import ffff9ada58b08000 refcount=4 obd=lustre-MDT0001-mdc-ffff9ada58b0a000 00000100:00000001:1.0:1788572368.249239:0:58077:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788572368.249240:0:58077:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788572368.249242:0:58077:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788572368.249244:0:58077:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada48e6f100. 00000100:00000001:1.0:1788572368.249247:0:58077:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572368.249250:0:58077:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID req@ffff9ada456bd880 x1875449616096512/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788572368.249257:0:58077:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788572368.249259:0:58077:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572368.249262:0:58077:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000082674ba1] to pc [ptlrpcd_00_01+1] req@ffff9ada456bd880 x1875449616096512/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572368.249269:0:58077:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572368.249271:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1788572368.249274:0:58077:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:1.0:1788572368.249277:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1788572368.249280:0:58077:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:1.0:1788572368.249285:0:58077:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (4290) 00000100:00000001:0.0:1788572368.249387:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788572368.249392:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788572368.249395:0:2384:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788572368.249403:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada456bed80 x1875449616096256/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572368.249411:0:2384:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788572368.249413:0:2384:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572368.249417:0:2384:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788572368.249420:0:2384:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788572368.249422:0:2384:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788572368.249426:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada456bed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449616096256:192.168.202.113@tcp:400:kworker.0 00000100:00000001:0.0:1788572368.249430:0:2384:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788572368.249437:0:2384:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788572368.249439:0:2384:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572368.249441:0:2384:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788572368.249447:0:2384:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada5170e200. 02000000:00000001:0.0:1788572368.249448:0:2384:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788572368.249457:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:0.0:1788572368.249462:0:2384:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada60acaf20. 00000400:00000010:0.0:1788572368.249480:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada5176aaa0. 00000100:00000200:0.0:1788572368.249489:0:2384:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616096256, portal 25 00000100:00000001:0.0:1788572368.249492:0:2384:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788572368.249493:0:2384:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860991090048 : -111212718461568 : ffff9ada456bed80) 00000100:00000040:0.0:1788572368.249499:0:2384:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada456bed80 x1875449616096256/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572384 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572368.249506:0:2384:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788572368.249509:0:2384:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:0.0:1788572368.249512:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada5176af68. 00000100:00000200:0.0:1788572368.249516:0:2384:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875449616096256, offset 0 00000400:00000200:0.0:1788572368.249524:0:2384:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:0.0:1788572368.249534:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:0.0:1788572368.249544:0:2384:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:0.0:1788572368.249550:0:2384:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788572368.249555:0:2384:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:0.0:1788572368.249559:0:2384:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788572368.249562:0:2384:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:0.0:1788572368.249567:0:2384:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22324:22324:256:4294967295] 192.168.202.113@tcp LPNI seq info [22320:22320:8:4294967295] 00000400:00000200:0.0:1788572368.249571:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:0.0:1788572368.249582:0:2384:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:0.0:1788572368.249588:0:2384:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:0.0:1788572368.249594:0:2384:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada5074a700. 00000800:00000200:0.0:1788572368.249601:0:2384:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:0.0:1788572368.249610:0:2384:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:0.0:1788572368.249614:0:2384:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada5074a700 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788572368.249644:0:2384:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788572368.249649:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:0.0:1788572368.249652:0:2384:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS new deadline estimation for cpu 2 is 4298963790 00000100:00000001:0.0:1788572368.249659:0:2384:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4298958669 00000100:00000001:0.0:1788572368.249663:0:2384:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS new deadline estimation for cpu 3 is 4298963790 00000100:00000001:0.0:1788572368.249667:0:2384:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4298958669 00000100:00000001:0.0:1788572368.249670:0:2384:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788572368.249672:0:2384:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572368.249676:0:2384:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788572368.249680:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada456bd880 x1875449616096512/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572368.249687:0:2384:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788572368.249688:0:2384:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572368.249690:0:2384:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788572368.249692:0:2384:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788572368.249694:0:2384:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788572368.249696:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada456bd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616096512:192.168.202.113@tcp:400:kworker.0 00000100:00000001:0.0:1788572368.249699:0:2384:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788572368.249701:0:2384:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788572368.249703:0:2384:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572368.249704:0:2384:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788572368.249708:0:2384:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada5170ee00. 02000000:00000001:0.0:1788572368.249709:0:2384:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788572368.249713:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:0.0:1788572368.249716:0:2384:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada60aca108. 00000400:00000010:0.0:1788572368.249720:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada5176add0. 00000100:00000200:0.0:1788572368.249732:0:2384:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616096512, portal 10 00000100:00000001:0.0:1788572368.249735:0:2384:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788572368.249736:0:2384:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860991084672 : -111212718466944 : ffff9ada456bd880) 00000100:00000040:0.0:1788572368.249743:0:2384:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada456bd880 x1875449616096512/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572384 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572368.249749:0:2384:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788572368.249751:0:2384:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:0.0:1788572368.249753:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada5176a440. 00000100:00000200:0.0:1788572368.249756:0:2384:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875449616096512, offset 0 00000400:00000200:0.0:1788572368.249760:0:2384:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:0.0:1788572368.249764:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:0.0:1788572368.249767:0:2384:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:0.0:1788572368.249769:0:2384:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788572368.249771:0:2384:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:0.0:1788572368.249773:0:2384:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788572368.249775:0:2384:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:0.0:1788572368.249778:0:2384:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22325:22325:255:4294967295] 192.168.202.113@tcp LPNI seq info [22321:22321:7:4294967295] 00000400:00000200:0.0:1788572368.249781:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:0.0:1788572368.249788:0:2384:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:0.0:1788572368.249792:0:2384:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:0.0:1788572368.249796:0:2384:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada5074a600. 00000800:00000200:0.0:1788572368.249800:0:2384:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:0.0:1788572368.249804:0:2384:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:0.0:1788572368.249806:0:2384:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada5074a600 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788572368.249810:0:2384:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572368.249814:0:2384:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: using a timeout of 16708 usecs (17 jiffies) 00000400:00000200:0.0:1788572368.249817:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:0.0:1788572368.249819:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298953235 00000100:00000001:0.0:1788572368.249821:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298963790 00000100:00000001:0.0:1788572368.249824:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298953235 00000100:00000001:0.0:1788572368.249826:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298963790 00000100:00000001:0.0:1788572368.249828:0:2384:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788572368.249830:0:2384:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572368.249832:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572368.249835:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572368.249843:0:2384:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788572368.249846:0:2384:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788572368.249848:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788572368.249850:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788572368.249851:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788572368.249895:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00000001:0.0:1788572368.249896:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572368.249899:0:2384:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788572368.249900:0:2384:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788572368.249902:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788572368.249903:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788572368.249904:0:2384:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788572368.249908:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada456bfb80 x1875449616096384/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572368.249915:0:2384:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788572368.249916:0:2384:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572368.249918:0:2384:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788572368.249919:0:2384:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788572368.249921:0:2384:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788572368.249924:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada456bfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616096384:192.168.202.113@tcp:400:kworker.0 00000100:00000001:0.0:1788572368.249927:0:2384:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788572368.249929:0:2384:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788572368.249931:0:2384:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788572368.249932:0:2384:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788572368.249935:0:2384:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada5170e000. 02000000:00000001:0.0:1788572368.249937:0:2384:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788572368.249940:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:0.0:1788572368.249942:0:2384:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada60acacb8. 00000400:00000010:0.0:1788572368.249946:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada5176a000. 00000100:00000200:0.0:1788572368.249948:0:2384:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616096384, portal 10 00000100:00000001:0.0:1788572368.249950:0:2384:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788572368.249951:0:2384:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860991093632 : -111212718457984 : ffff9ada456bfb80) 00000100:00000040:0.0:1788572368.249956:0:2384:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada456bfb80 x1875449616096384/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572384 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572368.249962:0:2384:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788572368.249965:0:2384:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:0.0:1788572368.249967:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada5176a220. 00000100:00000200:0.0:1788572368.249969:0:2384:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875449616096384, offset 0 00000400:00000200:0.0:1788572368.249974:0:2384:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:0.0:1788572368.249977:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:0.0:1788572368.249981:0:2384:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:0.0:1788572368.249983:0:2384:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788572368.249986:0:2384:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:0.0:1788572368.249987:0:2384:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788572368.249989:0:2384:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:0.0:1788572368.249992:0:2384:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22326:22326:254:4294967295] 192.168.202.113@tcp LPNI seq info [22322:22322:6:4294967295] 00000400:00000200:0.0:1788572368.249995:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:0.0:1788572368.250000:0:2384:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:0.0:1788572368.250004:0:2384:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:0.0:1788572368.250007:0:2384:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada5074a300. 00000800:00000200:0.0:1788572368.250010:0:2384:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:0.0:1788572368.250014:0:2384:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:0.0:1788572368.250016:0:2384:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada5074a300 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788572368.250019:0:2384:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572368.250022:0:2384:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: using a timeout of 41263 usecs (42 jiffies) 00000400:00000200:0.0:1788572368.250025:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:0.0:1788572368.250027:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298953261 00000100:00000001:0.0:1788572368.250029:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298963790 00000100:00000001:0.0:1788572368.250031:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298953261 00000100:00000001:0.0:1788572368.250033:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298963790 00000100:00000001:0.0:1788572368.250036:0:2384:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788572368.250037:0:2384:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572368.250039:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572368.250041:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788572368.250078:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788572368.250080:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788572368.252205:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788572368.252217:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada5074a700. 00000400:00000200:2.0:1788572368.252229:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572368.252240:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:2.0:1788572368.252250:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada5176af68 00000400:00000010:2.0:1788572368.252253:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada5176af68. 00000100:00000001:2.0:1788572368.252259:0:2378:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:2.0:1788572368.252271:0:2378:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada456bed80 x1875449616096256/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572384 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572368.252285:0:2378:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572368.252288:0:2378:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada456bed80 x1875449616096256/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572384 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572368.252296:0:2378:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572368.252298:0:2378:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000200:2.0:1788572368.252313:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788572368.252315:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada5074a600. 00000400:00000200:2.0:1788572368.252318:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572368.252322:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:2.0:1788572368.252324:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada5176a440 00000400:00000010:2.0:1788572368.252326:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada5176a440. 00000100:00000001:2.0:1788572368.252328:0:2378:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:2.0:1788572368.252332:0:2378:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada456bd880 x1875449616096512/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572384 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572368.252340:0:2378:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572368.252343:0:2378:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada456bd880 x1875449616096512/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572384 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572368.252349:0:2378:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572368.252350:0:2378:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000200:2.0:1788572368.252424:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788572368.252427:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada5074a300. 00000400:00000200:2.0:1788572368.252431:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572368.252435:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:2.0:1788572368.252438:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada5176a220 00000400:00000010:2.0:1788572368.252439:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada5176a220. 00000100:00000001:2.0:1788572368.252441:0:2378:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:2.0:1788572368.252446:0:2378:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada456bfb80 x1875449616096384/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572384 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572368.252452:0:2378:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572368.252455:0:2378:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada456bfb80 x1875449616096384/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572384 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572368.252460:0:2378:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572368.252461:0:2378:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000001:2.0:1788572368.256236:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572368.256271:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788572368.256277:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572368.256280:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572368.256291:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572368.256300:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:2.0:1788572368.256310:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572368.256315:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572368.256322:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572368.256327:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 10 MB=0x6a9b624295d00 00000400:00000200:2.0:1788572368.256333:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572368.256338:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.113@tcp of length 224/224 into md 0x358b5 [1] + 224 00000800:00000001:2.0:1788572368.256349:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572368.256352:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572368.256356:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572368.256361:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:2.0:1788572368.256370:0:2377:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:2.0:1788572368.256381:0:2377:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada456bd880 x1875449616096512/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572384 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788572368.256393:0:2377:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada456bd880 x1875449616096512/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572384 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572368.256425:0:2377:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788572368.256434:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572368.256437:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788572368.256438:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572368.256440:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572368.256443:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572368.256447:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:2.0:1788572368.256451:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572368.256454:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572368.256457:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572368.256459:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 10 MB=0x6a9b624295c80 00000400:00000200:2.0:1788572368.256463:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572368.256466:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.113@tcp of length 224/224 into md 0x358bd [1] + 224 00000800:00000001:2.0:1788572368.256470:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572368.256473:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572368.256475:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572368.256477:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:2.0:1788572368.256480:0:2377:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:2.0:1788572368.256484:0:2377:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada456bfb80 x1875449616096384/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572384 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788572368.256493:0:2377:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada456bfb80 x1875449616096384/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572384 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572368.256507:0:2377:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788572368.256510:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572368.256512:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788572368.256513:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572368.256515:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572368.256518:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572368.256521:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:2.0:1788572368.256524:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572368.256526:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572368.256529:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572368.256531:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 25 MB=0x6a9b624295c00 00000400:00000200:2.0:1788572368.256534:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572368.256538:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.202.113@tcp of length 224/224 into md 0x358ad [1] + 224 00000800:00000001:2.0:1788572368.256540:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572368.256546:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572368.256548:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572368.256550:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:2.0:1788572368.256553:0:2377:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:2.0:1788572368.256556:0:2377:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada456bed80 x1875449616096256/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572384 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788572368.256564:0:2377:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada456bed80 x1875449616096256/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572384 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572368.256571:0:2377:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788572368.256573:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572368.256575:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572368.260036:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572368.260048:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788572368.260062:0:2384:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788572368.260064:0:2384:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788572368.260070:0:2384:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada60acacb8. 00000400:00000200:1.0:1788572368.260082:0:2384:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada5176a000 00000400:00000010:1.0:1788572368.260085:0:2384:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada5176a000. 00000100:00000001:1.0:1788572368.260091:0:2384:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:1.0:1788572368.260106:0:2384:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada456bfb80 x1875449616096384/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572384 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788572368.260118:0:2384:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada456bfb80 x1875449616096384/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572384 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572368.260127:0:2384:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788572368.260129:0:2384:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572368.260132:0:2384:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1788572368.260136:0:2384:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788572368.260368:0:2384:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788572368.260371:0:2384:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572368.260375:0:2384:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788572368.260388:0:2384:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada51140460 time=63 v=5 (5 5 5 5) 00000100:00001000:1.0:1788572368.260394:0:2384:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada51140418 time=9 v=5 (1 2 0 0) 00000100:00000001:1.0:1788572368.260397:0:2384:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788572368.260400:0:2384:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572368.260404:0:2384:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788572368.260409:0:2384:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572368.260413:0:2384:0:(client.c:3110:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788572368.260415:0:2384:0:(client.c:3122:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: skip recheck: last_committed 219043334508 00000100:00000001:1.0:1788572368.260418:0:2384:0:(client.c:3123:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788572368.260419:0:2384:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572368.260426:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada456bfb80 x1875449616096384/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572384 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572368.260434:0:2384:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572368.260438:0:2384:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788572368.260440:0:2384:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788572368.260445:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada456bfb80 x1875449616096384/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572384 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572368.260453:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada456bfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616096384:192.168.202.113@tcp:400:kworker.0 00000400:00000001:1.0:1788572368.260461:0:2384:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788572368.260463:0:2384:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788572368.260465:0:2384:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada60acaf20. 00000400:00000200:1.0:1788572368.260468:0:2384:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada5176aaa0 00000400:00000010:1.0:1788572368.260481:0:2384:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada5176aaa0. 00000100:00000001:1.0:1788572368.260484:0:2384:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:1.0:1788572368.260488:0:2384:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada456bed80 x1875449616096256/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572384 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788572368.260498:0:2384:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada456bed80 x1875449616096256/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572384 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572368.260504:0:2384:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788572368.260505:0:2384:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572368.260507:0:2384:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1788572368.260509:0:2384:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788572368.260510:0:2384:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788572368.260512:0:2384:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572368.260514:0:2384:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788572368.260517:0:2384:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada58a62c60 time=130 v=5 (5 5 5 5) 00000100:00001000:1.0:1788572368.260520:0:2384:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada58a62c18 time=10 v=5 (1 1 0 0) 00000100:00000001:1.0:1788572368.260523:0:2384:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788572368.260524:0:2384:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572368.260526:0:2384:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788572368.260528:0:2384:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572368.260530:0:2384:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572368.260534:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada456bed80 x1875449616096256/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572384 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572368.260541:0:2384:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572368.260543:0:2384:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788572368.260545:0:2384:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788572368.260548:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada456bed80 x1875449616096256/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572384 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572368.260556:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada456bed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449616096256:192.168.202.113@tcp:400:kworker.0 00000400:00000001:1.0:1788572368.260562:0:2384:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788572368.260563:0:2384:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788572368.260565:0:2384:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada60aca108. 00000400:00000200:1.0:1788572368.260568:0:2384:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada5176add0 00000400:00000010:1.0:1788572368.260569:0:2384:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada5176add0. 00000100:00000001:1.0:1788572368.260572:0:2384:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:1.0:1788572368.260576:0:2384:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada456bd880 x1875449616096512/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572384 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788572368.260585:0:2384:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada456bd880 x1875449616096512/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572384 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572368.260592:0:2384:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788572368.260593:0:2384:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572368.260595:0:2384:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1788572368.260596:0:2384:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788572368.260597:0:2384:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788572368.260598:0:2384:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572368.260600:0:2384:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788572368.260605:0:2384:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada58b08460 time=130 v=5 (5 5 5 5) 00000100:00001000:1.0:1788572368.260607:0:2384:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada58b08418 time=130 v=5 (1 1 1 1) 00000100:00000001:1.0:1788572368.260609:0:2384:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788572368.260611:0:2384:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572368.260612:0:2384:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788572368.260614:0:2384:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572368.260616:0:2384:0:(client.c:3110:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788572368.260618:0:2384:0:(client.c:3122:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff9ada58b0a000: skip recheck: last_committed 0 00000100:00000001:1.0:1788572368.260619:0:2384:0:(client.c:3123:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788572368.260621:0:2384:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572368.260624:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada456bd880 x1875449616096512/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572384 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572368.260631:0:2384:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572368.260633:0:2384:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788572368.260645:0:2384:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788572368.260649:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada456bd880 x1875449616096512/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572384 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572368.260656:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada456bd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616096512:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572368.260660:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572368.260664:0:2384:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788572368.260668:0:2384:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada456bfb80 x1875449616096384/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572384 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572368.260675:0:2384:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788572368.260677:0:2384:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788572368.260681:0:2384:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada5170e000. 02000000:00000001:1.0:1788572368.260686:0:2384:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788572368.260690:0:2384:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788572368.260691:0:2384:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=3 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:1.0:1788572368.260694:0:2384:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1788572368.260697:0:2384:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada48e6ff00. 02000000:00000001:1.0:1788572368.260701:0:2384:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788572368.260703:0:2384:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788572368.260706:0:2384:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada456bfb80. 00000100:00000001:1.0:1788572368.260710:0:2384:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788572368.260712:0:2384:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572368.260714:0:2384:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788572368.260717:0:2384:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada456bed80 x1875449616096256/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572384 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572368.260724:0:2384:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788572368.260725:0:2384:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788572368.260727:0:2384:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada5170e200. 02000000:00000001:1.0:1788572368.260729:0:2384:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788572368.260730:0:2384:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788572368.260731:0:2384:0:(genops.c:1223:class_import_put()) import ffff9ada58a62800 refcount=3 obd=MGC192.168.202.113@tcp 00000020:00000001:1.0:1788572368.260733:0:2384:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1788572368.260734:0:2384:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada48e6f800. 02000000:00000001:1.0:1788572368.260736:0:2384:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788572368.260737:0:2384:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788572368.260738:0:2384:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada456bed80. 00000100:00000001:1.0:1788572368.260741:0:2384:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788572368.260743:0:2384:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572368.260744:0:2384:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788572368.260747:0:2384:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada456bd880 x1875449616096512/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572384 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572368.260754:0:2384:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788572368.260755:0:2384:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788572368.260756:0:2384:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada5170ee00. 02000000:00000001:1.0:1788572368.260758:0:2384:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788572368.260759:0:2384:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788572368.260760:0:2384:0:(genops.c:1223:class_import_put()) import ffff9ada58b08000 refcount=3 obd=lustre-MDT0001-mdc-ffff9ada58b0a000 00000020:00000001:1.0:1788572368.260762:0:2384:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1788572368.260763:0:2384:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada48e6f100. 02000000:00000001:1.0:1788572368.260765:0:2384:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788572368.260766:0:2384:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788572368.260767:0:2384:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada456bd880. 00000100:00000001:1.0:1788572368.260769:0:2384:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788572368.260770:0:2384:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572368.260773:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572368.260781:0:2384:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788572368.260784:0:2384:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572368.260786:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572368.260789:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572368.759035:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572368.759045:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572368.759053:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572368.759054:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572368.759057:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572368.759059:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788572368.759242:0:33423:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788572368.759246:0:33423:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788572368.759250:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572369.783821:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572369.783832:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572369.783839:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572369.783841:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572369.783844:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572369.783845:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572369.783967:0:15205:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572369.783971:0:15205:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572369.783974:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572370.808320:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572370.808331:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572370.808338:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572370.808340:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572370.808344:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572370.808345:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572370.808456:0:15204:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572370.808460:0:15204:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572370.808463:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572371.831047:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572371.831057:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572371.831064:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572371.831066:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572371.831069:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572371.831071:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788572371.831123:0:33423:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788572371.831127:0:33423:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788572371.831131:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572372.860053:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572372.860063:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572372.860070:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572372.860071:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572372.860075:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572372.860076:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572372.860158:0:15205:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572372.860162:0:15205:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572372.860166:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788572373.368554:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788572373.368567:0:77466:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788572373.368576:0:77466:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada45461180. 00000020:00000040:0.0:1788572373.368583:0:77466:0:(genops.c:1210:class_import_get()) import ffff9ada58a62800 refcount=4 obd=MGC192.168.202.113@tcp 00000100:00000001:0.0:1788572373.368589:0:77466:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788572373.368592:0:77466:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788572373.368598:0:77466:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788572373.368607:0:77466:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada5074a500. 00000100:00000001:0.0:1788572373.368615:0:77466:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572373.368629:0:77466:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS req@ffff9ada45461180 x1875449616096640/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788572373.368640:0:77466:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788572373.368646:0:77466:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572373.368653:0:77466:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ccc4131a] to pc [ptlrpcd_00_00+0] req@ffff9ada45461180 x1875449616096640/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572373.368684:0:77466:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572373.368687:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788572373.368691:0:77466:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788572373.368694:0:77466:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada45460700. 00000020:00000040:0.0:1788572373.368697:0:77466:0:(genops.c:1210:class_import_get()) import ffff9ada51140000 refcount=4 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000100:00000001:0.0:1788572373.368699:0:77466:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788572373.368701:0:77466:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788572373.368703:0:77466:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788572373.368707:0:77466:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada5074a200. 00000100:00000001:0.0:1788572373.368710:0:77466:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572373.368715:0:77466:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID req@ffff9ada45460700 x1875449616096768/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788572373.368722:0:77466:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788572373.368724:0:77466:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572373.368727:0:77466:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000000908cd5d] to pc [ptlrpcd_00_01+1] req@ffff9ada45460700 x1875449616096768/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572373.368746:0:77466:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572373.368748:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788572373.368752:0:77466:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788572373.368755:0:77466:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada45463100. 00000020:00000040:0.0:1788572373.368757:0:77466:0:(genops.c:1210:class_import_get()) import ffff9ada58b08000 refcount=4 obd=lustre-MDT0001-mdc-ffff9ada58b0a000 00000100:00000001:0.0:1788572373.368760:0:77466:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788572373.368761:0:77466:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788572373.368763:0:77466:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788572373.368766:0:77466:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada5074a000. 00000100:00000001:0.0:1788572373.368769:0:77466:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572373.368773:0:77466:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID req@ffff9ada45463100 x1875449616096896/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788572373.368786:0:77466:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788572373.368787:0:77466:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572373.368791:0:77466:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000602172c3] to pc [ptlrpcd_00_00+0] req@ffff9ada45463100 x1875449616096896/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572373.368798:0:77466:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572373.368800:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:0.0:1788572373.368804:0:77466:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:0.0:1788572373.368806:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:0.0:1788572373.368809:0:77466:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:0.0:1788572373.368814:0:77466:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (4296) 00000100:00000001:1.0:1788572373.369612:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572373.369620:0:2383:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788572373.369623:0:2383:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788572373.369634:0:2383:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada45461180 x1875449616096640/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572373.369644:0:2383:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788572373.369646:0:2383:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572373.369650:0:2383:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788572373.369653:0:2383:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788572373.369656:0:2383:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788572373.369659:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45461180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449616096640:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572373.369664:0:2383:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788572373.369670:0:2383:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788572373.369673:0:2383:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572373.369676:0:2383:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788572373.369682:0:2383:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada607e0800. 02000000:00000001:1.0:1788572373.369684:0:2383:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788572373.369693:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:1.0:1788572373.369697:0:2383:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada600d1c60. 00000400:00000010:1.0:1788572373.369706:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2908. 00000100:00000200:1.0:1788572373.369715:0:2383:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616096640, portal 25 00000100:00000001:1.0:1788572373.369718:0:2383:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788572373.369720:0:2383:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860988608896 : -111212720942720 : ffff9ada45461180) 00000100:00000040:1.0:1788572373.369726:0:2383:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada45461180 x1875449616096640/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572389 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572373.369750:0:2383:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788572373.369754:0:2383:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:1.0:1788572373.369757:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2088. 00000100:00000200:1.0:1788572373.369761:0:2383:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875449616096640, offset 0 00000400:00000200:1.0:1788572373.369769:0:2383:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:1.0:1788572373.369779:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572373.369788:0:2383:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:1.0:1788572373.369794:0:2383:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788572373.369799:0:2383:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:1.0:1788572373.369804:0:2383:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788572373.369806:0:2383:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:1.0:1788572373.369811:0:2383:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22327:22327:256:4294967295] 192.168.202.113@tcp LPNI seq info [22323:22323:8:4294967295] 00000400:00000200:1.0:1788572373.369815:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572373.369826:0:2383:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:1.0:1788572373.369833:0:2383:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:1.0:1788572373.369838:0:2383:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada48e6f100. 00000800:00000200:1.0:1788572373.369845:0:2383:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:1.0:1788572373.369882:0:2383:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:1.0:1788572373.369888:0:2383:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada48e6f100 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788572373.369900:0:2383:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788572373.369904:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:1.0:1788572373.369907:0:2383:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS new deadline estimation for cpu 2 is 4298968911 00000100:00000001:1.0:1788572373.369913:0:2383:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4298963790 00000100:00000001:1.0:1788572373.369917:0:2383:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS new deadline estimation for cpu 3 is 4298968911 00000100:00000001:1.0:1788572373.369921:0:2383:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4298963790 00000100:00000001:1.0:1788572373.369923:0:2383:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572373.369926:0:2383:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572373.369930:0:2383:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788572373.369934:0:2383:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada45463100 x1875449616096896/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572373.369939:0:2383:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788572373.369941:0:2383:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572373.369943:0:2383:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788572373.369945:0:2383:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788572373.369947:0:2383:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788572373.369950:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45463100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616096896:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572373.369953:0:2383:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788572373.369956:0:2383:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788572373.369958:0:2383:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572373.369959:0:2383:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788572373.369962:0:2383:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada607e0c00. 02000000:00000001:1.0:1788572373.369964:0:2383:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788572373.369967:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:1.0:1788572373.369970:0:2383:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada600d1ec8. 00000400:00000010:1.0:1788572373.369974:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2bb0. 00000100:00000200:1.0:1788572373.369976:0:2383:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616096896, portal 10 00000100:00000001:1.0:1788572373.369978:0:2383:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788572373.369979:0:2383:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860988616960 : -111212720934656 : ffff9ada45463100) 00000100:00000040:1.0:1788572373.369987:0:2383:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada45463100 x1875449616096896/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572389 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000800:00000200:3.0:1788572373.369992:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1788572373.369993:0:2383:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788572373.369995:0:2383:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:1.0:1788572373.369997:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2880. 00000800:00000010:3.0:1788572373.369997:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada48e6f100. 00000100:00000200:1.0:1788572373.369999:0:2383:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875449616096896, offset 0 00000400:00000200:3.0:1788572373.370003:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788572373.370004:0:2383:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:1.0:1788572373.370007:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572373.370009:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:1.0:1788572373.370010:0:2383:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:1.0:1788572373.370013:0:2383:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788572373.370015:0:2383:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:3.0:1788572373.370016:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2088 00000400:00000200:1.0:1788572373.370017:0:2383:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000010:3.0:1788572373.370019:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2088. 00000400:00000200:1.0:1788572373.370020:0:2383:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:1.0:1788572373.370022:0:2383:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22328:22328:255:4294967295] 192.168.202.113@tcp LPNI seq info [22324:22324:7:4294967295] 00000400:00000200:1.0:1788572373.370026:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:3.0:1788572373.370026:0:2378:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:3.0:1788572373.370030:0:2378:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada45461180 x1875449616096640/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572389 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1788572373.370033:0:2383:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:1.0:1788572373.370037:0:2383:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:1.0:1788572373.370040:0:2383:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada48e6f800. 00000100:00000001:3.0:1788572373.370041:0:2378:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000800:00000200:1.0:1788572373.370044:0:2383:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000100:00000040:3.0:1788572373.370046:0:2378:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada45461180 x1875449616096640/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572389 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000800:00000200:1.0:1788572373.370048:0:2383:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:1.0:1788572373.370051:0:2383:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada48e6f800 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788572373.370053:0:2378:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572373.370054:0:2383:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572373.370055:0:2378:0:(events.c:69:request_out_callback()) Process leaving 00000100:00000040:1.0:1788572373.370058:0:2383:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: using a timeout of 16706 usecs (17 jiffies) 00000400:00000200:1.0:1788572373.370060:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:1.0:1788572373.370062:0:2383:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298958356 00000100:00000001:1.0:1788572373.370065:0:2383:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298968911 00000100:00000001:1.0:1788572373.370066:0:2383:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298958356 00000100:00000001:1.0:1788572373.370068:0:2383:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298968911 00000100:00000001:1.0:1788572373.370070:0:2383:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572373.370072:0:2383:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788572373.370072:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788572373.370074:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada48e6f800. 00000100:00000001:1.0:1788572373.370075:0:2383:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788572373.370076:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1788572373.370078:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572373.370079:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:3.0:1788572373.370082:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2880 00000400:00000010:3.0:1788572373.370084:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2880. 00000100:00000001:3.0:1788572373.370086:0:2378:0:(events.c:35:request_out_callback()) Process entered 00000100:00000001:1.0:1788572373.370087:0:2383:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000200:3.0:1788572373.370088:0:2378:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada45463100 x1875449616096896/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572389 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572373.370089:0:2383:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788572373.370092:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572373.370093:0:2383:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788572373.370095:0:2383:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572373.370096:0:2378:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00100000:1.0:1788572373.370097:0:2383:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00000001:1.0:1788572373.370099:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788572373.370099:0:2378:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada45463100 x1875449616096896/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572389 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572373.370101:0:2383:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788572373.370103:0:2383:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572373.370105:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788572373.370105:0:2378:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572373.370106:0:2383:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788572373.370107:0:2383:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1788572373.370107:0:2378:0:(events.c:69:request_out_callback()) Process leaving 00000100:00000040:1.0:1788572373.370111:0:2383:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada45460700 x1875449616096768/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572373.370117:0:2383:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788572373.370119:0:2383:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572373.370121:0:2383:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788572373.370122:0:2383:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788572373.370124:0:2383:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788572373.370128:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45460700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616096768:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572373.370131:0:2383:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788572373.370133:0:2383:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788572373.370135:0:2383:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572373.370136:0:2383:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788572373.370139:0:2383:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada607e0a00. 02000000:00000001:1.0:1788572373.370141:0:2383:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788572373.370143:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:1.0:1788572373.370146:0:2383:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada600d16e0. 00000400:00000010:1.0:1788572373.370148:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2110. 00000100:00000200:1.0:1788572373.370151:0:2383:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616096768, portal 10 00000100:00000001:1.0:1788572373.370153:0:2383:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788572373.370154:0:2383:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860988606208 : -111212720945408 : ffff9ada45460700) 00000100:00000040:1.0:1788572373.370160:0:2383:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada45460700 x1875449616096768/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572389 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572373.370165:0:2383:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788572373.370168:0:2383:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:1.0:1788572373.370170:0:2383:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d27f8. 00000100:00000200:1.0:1788572373.370173:0:2383:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875449616096768, offset 0 00000400:00000200:1.0:1788572373.370177:0:2383:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:1.0:1788572373.370181:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572373.370184:0:2383:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:1.0:1788572373.370187:0:2383:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788572373.370189:0:2383:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:1.0:1788572373.370191:0:2383:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788572373.370193:0:2383:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:1.0:1788572373.370196:0:2383:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22329:22329:256:4294967295] 192.168.202.113@tcp LPNI seq info [22325:22325:8:4294967295] 00000400:00000200:1.0:1788572373.370200:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572373.370205:0:2383:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:1.0:1788572373.370210:0:2383:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:1.0:1788572373.370213:0:2383:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada48e6ff00. 00000800:00000200:1.0:1788572373.370217:0:2383:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:1.0:1788572373.370223:0:2383:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:1.0:1788572373.370225:0:2383:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada48e6ff00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788572373.370243:0:2383:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572373.370247:0:2383:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: using a timeout of 41262 usecs (42 jiffies) 00000400:00000200:1.0:1788572373.370250:0:2383:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:1.0:1788572373.370251:0:2383:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298958381 00000100:00000001:1.0:1788572373.370254:0:2383:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298968911 00000100:00000001:1.0:1788572373.370256:0:2383:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298958381 00000100:00000001:1.0:1788572373.370258:0:2383:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298968911 00000100:00000001:1.0:1788572373.370260:0:2383:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572373.370262:0:2383:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572373.370265:0:2383:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572373.370267:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572373.370278:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572373.370281:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788572373.372023:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572373.372043:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572373.372049:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572373.372052:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572373.372061:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572373.372069:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572373.372079:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572373.372084:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572373.372089:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572373.372094:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 25 MB=0x6a9b624295d80 00000400:00000200:3.0:1788572373.372100:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572373.372105:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.202.113@tcp of length 224/224 into md 0x358c5 [1] + 224 00000800:00000001:3.0:1788572373.372114:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572373.372131:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572373.372135:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572373.372142:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572373.372151:0:2377:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572373.372159:0:2377:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada45461180 x1875449616096640/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572389 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572373.372170:0:2377:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada45461180 x1875449616096640/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572389 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572373.372193:0:2377:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788572373.372215:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000200:3.0:1788572373.372370:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788572373.372374:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada48e6ff00. 00000400:00000200:3.0:1788572373.372380:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572373.372384:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:3.0:1788572373.372388:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d27f8 00000400:00000010:3.0:1788572373.372391:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d27f8. 00000100:00000001:3.0:1788572373.372397:0:2377:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:3.0:1788572373.372402:0:2377:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada45460700 x1875449616096768/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572389 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572373.372414:0:2377:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572373.372418:0:2377:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada45460700 x1875449616096768/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572389 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572373.372425:0:2377:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572373.372427:0:2377:0:(events.c:69:request_out_callback()) Process leaving 00000100:00000001:1.0:1788572373.373252:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572373.373259:0:2383:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788572373.373268:0:2383:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788572373.373280:0:2383:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788572373.373285:0:2383:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada600d1c60. 00000400:00000200:1.0:1788572373.373292:0:2383:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2908 00000400:00000010:1.0:1788572373.373294:0:2383:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2908. 00000100:00000001:1.0:1788572373.373298:0:2383:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:1.0:1788572373.373306:0:2383:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada45461180 x1875449616096640/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572389 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788572373.373316:0:2383:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada45461180 x1875449616096640/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572389 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572373.373322:0:2383:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788572373.373324:0:2383:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572373.373327:0:2383:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1788572373.373331:0:2383:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788572373.373334:0:2383:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788572373.373337:0:2383:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572373.373340:0:2383:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788572373.373349:0:2383:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada58a62c60 time=135 v=5 (5 5 5 5) 00000100:00001000:1.0:1788572373.373353:0:2383:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada58a62c18 time=15 v=5 (1 1 0 0) 00000100:00000001:1.0:1788572373.373361:0:2383:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788572373.373363:0:2383:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572373.373367:0:2383:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788572373.373369:0:2383:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572373.373372:0:2383:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572373.373376:0:2383:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada45461180 x1875449616096640/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572389 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572373.373383:0:2383:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572373.373386:0:2383:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788572373.373388:0:2383:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788572373.373391:0:2383:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada45461180 x1875449616096640/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572389 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572373.373398:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45461180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449616096640:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572373.373403:0:2383:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572373.373405:0:2383:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788572373.373408:0:2383:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45461180 x1875449616096640/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572389 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572373.373413:0:2383:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788572373.373416:0:2383:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788572373.373420:0:2383:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada607e0800. 02000000:00000001:1.0:1788572373.373423:0:2383:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788572373.373426:0:2383:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788572373.373428:0:2383:0:(genops.c:1223:class_import_put()) import ffff9ada58a62800 refcount=3 obd=MGC192.168.202.113@tcp 00000020:00000001:1.0:1788572373.373431:0:2383:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1788572373.373434:0:2383:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada5074a500. 02000000:00000001:1.0:1788572373.373438:0:2383:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788572373.373441:0:2383:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788572373.373443:0:2383:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45461180. 00000100:00000001:1.0:1788572373.373447:0:2383:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788572373.373448:0:2383:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572373.373452:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572373.373453:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572373.373454:0:2383:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788572373.373456:0:2383:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572373.373457:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788572373.373998:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788572373.376832:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572373.376883:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788572373.376888:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572373.376892:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572373.376903:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572373.376913:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:2.0:1788572373.376923:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572373.376929:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572373.376935:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572373.376941:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 10 MB=0x6a9b624295e80 00000400:00000200:2.0:1788572373.376946:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572373.376953:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.113@tcp of length 224/224 into md 0x358cd [1] + 224 00000800:00000001:2.0:1788572373.376962:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572373.376967:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572373.376970:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572373.376975:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:2.0:1788572373.376996:0:2377:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:2.0:1788572373.377007:0:2377:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada45463100 x1875449616096896/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572389 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788572373.377019:0:2377:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada45463100 x1875449616096896/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572389 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572373.377032:0:2377:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788572373.377041:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572373.377045:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788572373.377047:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572373.377049:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572373.377052:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572373.377055:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:2.0:1788572373.377059:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572373.377062:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572373.377065:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572373.377068:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 10 MB=0x6a9b624295e00 00000400:00000200:2.0:1788572373.377071:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572373.377075:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.113@tcp of length 224/224 into md 0x358d5 [1] + 224 00000800:00000001:2.0:1788572373.377079:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:1.0:1788572373.377578:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572373.377584:0:2383:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788572373.377593:0:2383:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788572373.377595:0:2383:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788572373.377598:0:2383:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada600d1ec8. 00000400:00000200:1.0:1788572373.377605:0:2383:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2bb0 00000400:00000010:1.0:1788572373.377608:0:2383:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2bb0. 00000100:00000001:1.0:1788572373.377612:0:2383:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:1.0:1788572373.377618:0:2383:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada45463100 x1875449616096896/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572389 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788572373.377626:0:2383:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada45463100 x1875449616096896/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572389 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572373.377632:0:2383:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788572373.377634:0:2383:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572373.377636:0:2383:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1788572373.377640:0:2383:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788572373.377643:0:2383:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788572373.377645:0:2383:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572373.377648:0:2383:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788572373.377660:0:2383:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada58b08460 time=135 v=5 (5 5 5 5) 00000100:00001000:1.0:1788572373.377664:0:2383:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada58b08418 time=135 v=5 (1 1 1 1) 00000100:00000001:1.0:1788572373.377667:0:2383:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788572373.377669:0:2383:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572373.377673:0:2383:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788572373.377676:0:2383:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572373.377680:0:2383:0:(client.c:3110:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788572373.377682:0:2383:0:(client.c:3122:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff9ada58b0a000: skip recheck: last_committed 0 00000100:00000001:1.0:1788572373.377684:0:2383:0:(client.c:3123:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788572373.377686:0:2383:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572373.377691:0:2383:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada45463100 x1875449616096896/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572389 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572373.377704:0:2383:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572373.377707:0:2383:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788572373.377709:0:2383:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788572373.377713:0:2383:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada45463100 x1875449616096896/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572389 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572373.377723:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45463100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616096896:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572373.377728:0:2383:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572373.377731:0:2383:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788572373.377735:0:2383:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45463100 x1875449616096896/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572389 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572373.377753:0:2383:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788572373.377755:0:2383:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788572373.377759:0:2383:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada607e0c00. 02000000:00000001:1.0:1788572373.377763:0:2383:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788572373.377766:0:2383:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788572373.377768:0:2383:0:(genops.c:1223:class_import_put()) import ffff9ada58b08000 refcount=3 obd=lustre-MDT0001-mdc-ffff9ada58b0a000 00000020:00000001:1.0:1788572373.377771:0:2383:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1788572373.377774:0:2383:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada5074a000. 02000000:00000001:1.0:1788572373.377778:0:2383:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788572373.377780:0:2383:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788572373.377782:0:2383:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45463100. 00000100:00000001:1.0:1788572373.377786:0:2383:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788572373.377787:0:2383:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572373.377791:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572373.377793:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572373.377794:0:2383:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788572373.377796:0:2383:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572373.377797:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788572373.378035:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572373.378039:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572373.378044:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:2.0:1788572373.378050:0:2377:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:2.0:1788572373.378055:0:2377:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada45460700 x1875449616096768/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572389 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788572373.378064:0:2377:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada45460700 x1875449616096768/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572389 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572373.378074:0:2377:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788572373.378081:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572373.378085:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572373.378791:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572373.378795:0:2383:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788572373.378801:0:2383:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788572373.378802:0:2383:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788572373.378806:0:2383:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada600d16e0. 00000400:00000200:1.0:1788572373.378810:0:2383:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2110 00000400:00000010:1.0:1788572373.378813:0:2383:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2110. 00000100:00000001:1.0:1788572373.378817:0:2383:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:1.0:1788572373.378823:0:2383:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada45460700 x1875449616096768/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572389 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788572373.378833:0:2383:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada45460700 x1875449616096768/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572389 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572373.378839:0:2383:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788572373.378840:0:2383:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572373.378842:0:2383:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1788572373.378844:0:2383:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788572373.378846:0:2383:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788572373.378848:0:2383:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572373.378850:0:2383:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788572373.378891:0:2383:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada51140460 time=68 v=5 (5 5 5 5) 00000100:00001000:1.0:1788572373.378894:0:2383:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada51140418 time=14 v=5 (1 2 0 0) 00000100:00000001:1.0:1788572373.378896:0:2383:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788572373.378898:0:2383:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572373.378900:0:2383:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788572373.378903:0:2383:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572373.378905:0:2383:0:(client.c:3110:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788572373.378907:0:2383:0:(client.c:3122:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: skip recheck: last_committed 219043334508 00000100:00000001:1.0:1788572373.378909:0:2383:0:(client.c:3123:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788572373.378910:0:2383:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572373.378914:0:2383:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada45460700 x1875449616096768/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572389 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572373.378920:0:2383:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572373.378922:0:2383:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788572373.378924:0:2383:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788572373.378927:0:2383:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada45460700 x1875449616096768/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572389 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572373.378934:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45460700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616096768:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572373.378940:0:2383:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572373.378942:0:2383:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788572373.378946:0:2383:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45460700 x1875449616096768/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572389 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572373.378953:0:2383:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788572373.378955:0:2383:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788572373.378958:0:2383:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada607e0a00. 02000000:00000001:1.0:1788572373.378962:0:2383:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788572373.378964:0:2383:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788572373.378965:0:2383:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=3 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:1.0:1788572373.378968:0:2383:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1788572373.378970:0:2383:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada5074a200. 02000000:00000001:1.0:1788572373.378973:0:2383:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788572373.378975:0:2383:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788572373.378977:0:2383:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45460700. 00000100:00000001:1.0:1788572373.378980:0:2383:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788572373.378981:0:2383:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572373.378994:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572373.379003:0:2383:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788572373.379005:0:2383:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572373.379008:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572373.379010:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572373.881172:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572373.881180:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572373.881187:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572373.881189:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572373.881193:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572373.881195:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572373.881263:0:15204:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572373.881267:0:15204:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572373.881270:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572374.903271:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572374.903282:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572374.903290:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572374.903291:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572374.903295:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572374.903296:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788572374.903422:0:33423:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788572374.903426:0:33423:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788572374.903430:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572375.930110:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572375.930120:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572375.930129:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572375.930133:0:365:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:2.0:1788572375.930136:0:365:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:2.0:1788572375.930140:0:365:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572375.930144:0:365:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:2.0:1788572375.930147:0:365:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572375.930149:0:365:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572375.930151:0:365:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572375.930155:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572375.930156:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572375.930328:0:15205:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572375.930333:0:15205:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572375.930336:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572376.952472:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572376.952482:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572376.952489:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572376.952491:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572376.952495:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572376.952496:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572376.952616:0:15204:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572376.952622:0:15204:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572376.952626:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572377.978278:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572377.978289:0:365:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:2.0:1788572377.978292:0:365:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:2.0:1788572377.978298:0:365:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572377.978304:0:365:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:2.0:1788572377.978307:0:365:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572377.978309:0:365:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572377.978311:0:365:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572377.978316:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572377.978318:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572377.978322:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572377.978324:0:365:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:2.0:1788572377.978325:0:365:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:2.0:1788572377.978330:0:365:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:2.0:1788572377.978333:0:365:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:2.0:1788572377.978335:0:365:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572377.978337:0:365:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:2.0:1788572377.978338:0:365:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572377.978340:0:365:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572377.978342:0:365:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788572377.979176:0:33423:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788572377.979182:0:33423:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788572377.979187:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:3.0:1788572378.487174:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788572378.487189:0:77466:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788572378.487203:0:77466:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada455a5880. 00000020:00000040:3.0:1788572378.487212:0:77466:0:(genops.c:1210:class_import_get()) import ffff9ada58a62800 refcount=4 obd=MGC192.168.202.113@tcp 00000100:00000001:3.0:1788572378.487217:0:77466:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788572378.487221:0:77466:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788572378.487228:0:77466:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788572378.487238:0:77466:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada50c31700. 00000100:00000001:3.0:1788572378.487247:0:77466:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572378.487263:0:77466:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS req@ffff9ada455a5880 x1875449616097024/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788572378.487276:0:77466:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788572378.487281:0:77466:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572378.487288:0:77466:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000fd71f97f] to pc [ptlrpcd_01_01+1] req@ffff9ada455a5880 x1875449616097024/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572378.487315:0:77466:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572378.487318:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788572378.487322:0:77466:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788572378.487325:0:77466:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada455a7b80. 00000020:00000040:3.0:1788572378.487328:0:77466:0:(genops.c:1210:class_import_get()) import ffff9ada51140000 refcount=4 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000100:00000001:3.0:1788572378.487330:0:77466:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788572378.487331:0:77466:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788572378.487333:0:77466:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788572378.487336:0:77466:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada50c31600. 00000100:00000001:3.0:1788572378.487339:0:77466:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572378.487343:0:77466:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID req@ffff9ada455a7b80 x1875449616097152/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788572378.487351:0:77466:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788572378.487353:0:77466:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572378.487356:0:77466:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000a5ad6ad4] to pc [ptlrpcd_01_00+0] req@ffff9ada455a7b80 x1875449616097152/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572378.487365:0:77466:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572378.487367:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788572378.487370:0:77466:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788572378.487373:0:77466:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada455a7480. 00000020:00000040:3.0:1788572378.487376:0:77466:0:(genops.c:1210:class_import_get()) import ffff9ada58b08000 refcount=4 obd=lustre-MDT0001-mdc-ffff9ada58b0a000 00000100:00000001:3.0:1788572378.487378:0:77466:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788572378.487379:0:77466:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788572378.487381:0:77466:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788572378.487384:0:77466:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada50c31500. 00000100:00000001:3.0:1788572378.487386:0:77466:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572378.487390:0:77466:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID req@ffff9ada455a7480 x1875449616097280/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788572378.487395:0:77466:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788572378.487397:0:77466:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572378.487400:0:77466:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000a4ba8365] to pc [ptlrpcd_01_01+1] req@ffff9ada455a7480 x1875449616097280/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572378.487407:0:77466:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572378.487409:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:3.0:1788572378.487412:0:77466:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:3.0:1788572378.487415:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:3.0:1788572378.487418:0:77466:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:3.0:1788572378.487423:0:77466:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (4301) 00000100:00000001:3.0:1788572378.487433:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788572378.487437:0:2386:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788572378.487439:0:2386:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788572378.487445:0:2386:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada455a5880 x1875449616097024/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572378.487453:0:2386:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788572378.487456:0:2386:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788572378.487460:0:2386:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788572378.487463:0:2386:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788572378.487466:0:2386:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788572378.487469:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449616097024:192.168.202.113@tcp:400:kworker.0 00000100:00000001:3.0:1788572378.487476:0:2386:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:2.0:1788572378.487643:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788572378.487646:0:2385:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788572378.487648:0:2385:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1788572378.487654:0:2385:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada455a7b80 x1875449616097152/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572378.487660:0:2385:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1788572378.487661:0:2385:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572378.487664:0:2385:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1788572378.487665:0:2385:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1788572378.487667:0:2385:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1788572378.487669:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616097152:192.168.202.113@tcp:400:kworker.0 00000100:00000001:2.0:1788572378.487673:0:2385:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1788572378.487677:0:2385:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1788572378.487679:0:2385:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572378.487682:0:2385:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1788572378.487689:0:2385:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada45574400. 02000000:00000001:2.0:1788572378.487690:0:2385:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1788572378.487699:0:2385:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:2.0:1788572378.487704:0:2385:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada43d96cb8. 00000400:00000010:2.0:1788572378.487713:0:2385:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada43c8fe58. 00000100:00000200:2.0:1788572378.487721:0:2385:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616097152, portal 10 00000100:00000001:2.0:1788572378.487724:0:2385:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1788572378.487726:0:2385:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860989946752 : -111212719604864 : ffff9ada455a7b80) 00000100:00000040:2.0:1788572378.487735:0:2385:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada455a7b80 x1875449616097152/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572394 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572378.487744:0:2385:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1788572378.487747:0:2385:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:2.0:1788572378.487750:0:2385:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada43c8fc38. 00000100:00000200:2.0:1788572378.487753:0:2385:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875449616097152, offset 0 00000400:00000200:2.0:1788572378.487761:0:2385:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:2.0:1788572378.487771:0:2385:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572378.487780:0:2385:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:2.0:1788572378.487786:0:2385:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1788572378.487791:0:2385:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:2.0:1788572378.487795:0:2385:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1788572378.487799:0:2385:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:2.0:1788572378.487804:0:2385:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22330:22330:256:4294967295] 192.168.202.113@tcp LPNI seq info [22326:22326:8:4294967295] 00000400:00000200:2.0:1788572378.487809:0:2385:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572378.487820:0:2385:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:2.0:1788572378.487826:0:2385:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:2.0:1788572378.487832:0:2385:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada509edf00. 00000800:00000200:2.0:1788572378.487890:0:2385:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:2.0:1788572378.487899:0:2385:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:2.0:1788572378.487905:0:2385:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada509edf00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1788572378.487916:0:2385:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572378.487921:0:2385:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: using a timeout of 41261 usecs (42 jiffies) 00000400:00000200:2.0:1788572378.487924:0:2385:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:2.0:1788572378.487928:0:2385:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298963498 00000100:00000001:2.0:1788572378.487931:0:2385:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298968911 00000100:00000001:2.0:1788572378.487934:0:2385:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298963498 00000100:00000001:2.0:1788572378.487936:0:2385:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298968911 00000100:00000001:2.0:1788572378.487938:0:2385:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788572378.487941:0:2385:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572378.487945:0:2385:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572378.487948:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572378.487957:0:2385:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788572378.487959:0:2385:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1788572378.487961:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788572378.487963:0:2385:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788572378.487965:0:2385:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572378.487967:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788572378.488099:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788572378.488103:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada509edf00. 00000400:00000200:2.0:1788572378.488109:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572378.488115:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:2.0:1788572378.488122:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada43c8fc38 00000400:00000010:2.0:1788572378.488125:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada43c8fc38. 00000100:00000001:2.0:1788572378.488129:0:2378:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:2.0:1788572378.488134:0:2378:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada455a7b80 x1875449616097152/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572394 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572378.488145:0:2378:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572378.488148:0:2378:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada455a7b80 x1875449616097152/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572394 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572378.488156:0:2378:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572378.488158:0:2378:0:(events.c:69:request_out_callback()) Process leaving 02000000:00000001:3.0:1788572378.488559:0:2386:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788572378.488562:0:2386:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788572378.488565:0:2386:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788572378.488572:0:2386:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada46490400. 02000000:00000001:3.0:1788572378.488575:0:2386:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788572378.488581:0:2386:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:3.0:1788572378.488584:0:2386:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada510c98f0. 00000400:00000010:3.0:1788572378.488590:0:2386:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada51269110. 00000100:00000200:3.0:1788572378.488595:0:2386:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616097024, portal 25 00000100:00000001:3.0:1788572378.488597:0:2386:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788572378.488599:0:2386:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860989937792 : -111212719613824 : ffff9ada455a5880) 00000100:00000040:3.0:1788572378.488606:0:2386:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada455a5880 x1875449616097024/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572394 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572378.488613:0:2386:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788572378.488617:0:2386:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:3.0:1788572378.488619:0:2386:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada51269330. 00000100:00000200:3.0:1788572378.488622:0:2386:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875449616097024, offset 0 00000400:00000200:3.0:1788572378.488628:0:2386:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:3.0:1788572378.488633:0:2386:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572378.488638:0:2386:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:3.0:1788572378.488643:0:2386:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788572378.488646:0:2386:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:3.0:1788572378.488649:0:2386:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788572378.488651:0:2386:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:3.0:1788572378.488654:0:2386:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22331:22331:256:4294967295] 192.168.202.113@tcp LPNI seq info [22327:22327:8:4294967295] 00000400:00000200:3.0:1788572378.488659:0:2386:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572378.488666:0:2386:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:3.0:1788572378.488671:0:2386:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:3.0:1788572378.488674:0:2386:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada50c31000. 00000800:00000200:3.0:1788572378.488679:0:2386:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:3.0:1788572378.488685:0:2386:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:3.0:1788572378.488688:0:2386:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada50c31000 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788572378.488700:0:2386:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788572378.488703:0:2386:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:3.0:1788572378.488705:0:2386:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS new deadline estimation for cpu 2 is 4298974029 00000100:00000001:3.0:1788572378.488711:0:2386:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4298968911 00000100:00000001:3.0:1788572378.488716:0:2386:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS new deadline estimation for cpu 3 is 4298974029 00000100:00000001:3.0:1788572378.488719:0:2386:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4298968911 00000100:00000001:3.0:1788572378.488722:0:2386:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788572378.488724:0:2386:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572378.488727:0:2386:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788572378.488731:0:2386:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada455a7480 x1875449616097280/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572378.488740:0:2386:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788572378.488741:0:2386:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788572378.488744:0:2386:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788572378.488747:0:2386:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788572378.488749:0:2386:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788572378.488752:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada455a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616097280:192.168.202.113@tcp:400:kworker.0 00000100:00000001:3.0:1788572378.488756:0:2386:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788572378.488759:0:2386:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788572378.488761:0:2386:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788572378.488762:0:2386:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788572378.488766:0:2386:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada46491c00. 02000000:00000001:3.0:1788572378.488768:0:2386:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788572378.488771:0:2386:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:3.0:1788572378.488774:0:2386:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada510c9948. 00000400:00000010:3.0:1788572378.488777:0:2386:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada51269550. 00000100:00000200:3.0:1788572378.488780:0:2386:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616097280, portal 10 00000100:00000001:3.0:1788572378.488782:0:2386:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788572378.488783:0:2386:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860989944960 : -111212719606656 : ffff9ada455a7480) 00000100:00000040:3.0:1788572378.488790:0:2386:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada455a7480 x1875449616097280/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572394 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572378.488797:0:2386:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788572378.488800:0:2386:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:3.0:1788572378.488803:0:2386:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada51269aa0. 00000100:00000200:3.0:1788572378.488805:0:2386:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875449616097280, offset 0 00000400:00000200:3.0:1788572378.488809:0:2386:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:3.0:1788572378.488813:0:2386:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572378.488816:0:2386:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:3.0:1788572378.488819:0:2386:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788572378.488822:0:2386:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:3.0:1788572378.488824:0:2386:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788572378.488827:0:2386:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:3.0:1788572378.488830:0:2386:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22332:22332:255:4294967295] 192.168.202.113@tcp LPNI seq info [22328:22328:7:4294967295] 00000400:00000200:3.0:1788572378.488834:0:2386:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572378.488868:0:2386:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:3.0:1788572378.488873:0:2386:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:3.0:1788572378.488877:0:2386:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada50c31300. 00000800:00000200:3.0:1788572378.488881:0:2386:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:3.0:1788572378.488885:0:2386:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:3.0:1788572378.488888:0:2386:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada50c31300 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788572378.488891:0:2386:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572378.488894:0:2386:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: using a timeout of 16700 usecs (17 jiffies) 00000400:00000200:3.0:1788572378.488898:0:2386:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:3.0:1788572378.488900:0:2386:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298963475 00000100:00000001:3.0:1788572378.488902:0:2386:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298974029 00000100:00000001:3.0:1788572378.488905:0:2386:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298963475 00000100:00000001:3.0:1788572378.488907:0:2386:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298974029 00000100:00000001:3.0:1788572378.488909:0:2386:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788572378.488912:0:2386:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572378.488914:0:2386:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572378.488917:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788572378.488922:0:2386:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788572378.488924:0:2386:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788572378.488926:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788572378.488928:0:2386:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788572378.488929:0:2386:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572378.488931:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788572378.488960:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788572378.488963:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada50c31000. 00000400:00000200:3.0:1788572378.488967:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572378.488971:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:3.0:1788572378.488975:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada51269330 00000400:00000010:3.0:1788572378.488977:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada51269330. 00000100:00000001:3.0:1788572378.488981:0:2377:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:3.0:1788572378.488986:0:2377:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada455a5880 x1875449616097024/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572394 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572378.488995:0:2377:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572378.488998:0:2377:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada455a5880 x1875449616097024/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572394 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572378.489004:0:2377:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572378.489006:0:2377:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000200:3.0:1788572378.489054:0:2377:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788572378.489057:0:2377:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada50c31300. 00000400:00000200:3.0:1788572378.489060:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572378.489417:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:3.0:1788572378.489423:0:2377:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada51269aa0 00000400:00000010:3.0:1788572378.489425:0:2377:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada51269aa0. 00000100:00000001:3.0:1788572378.489428:0:2377:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:3.0:1788572378.489434:0:2377:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada455a7480 x1875449616097280/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572394 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572378.489443:0:2377:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572378.489447:0:2377:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada455a7480 x1875449616097280/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572394 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572378.489460:0:2377:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572378.489461:0:2377:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000001:2.0:1788572378.493754:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572378.493779:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788572378.493786:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572378.493789:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572378.493799:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572378.493810:0:2378:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:2.0:1788572378.493821:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572378.493829:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572378.493897:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572378.493903:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 10 MB=0x6a9b624295f80 00000400:00000200:2.0:1788572378.493909:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572378.493916:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.113@tcp of length 224/224 into md 0x358dd [1] + 224 00000800:00000001:2.0:1788572378.493924:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572378.493930:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572378.493934:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572378.493939:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:2.0:1788572378.493949:0:2378:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:2.0:1788572378.493962:0:2378:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada455a7b80 x1875449616097152/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572394 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788572378.493973:0:2378:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada455a7b80 x1875449616097152/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572394 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572378.494004:0:2378:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788572378.494015:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572378.494020:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788572378.494022:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572378.494024:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572378.494027:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572378.494031:0:2378:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:2.0:1788572378.494035:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572378.494038:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572378.494041:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572378.494044:0:2378:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 10 MB=0x6a9b624296000 00000400:00000200:2.0:1788572378.494047:0:2378:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:2.0:1788572378.494051:0:2378:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.113@tcp of length 224/224 into md 0x358fd [1] + 224 00000800:00000001:2.0:1788572378.494056:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572378.494134:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788572378.494137:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572378.494141:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:2.0:1788572378.494144:0:2378:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:2.0:1788572378.494150:0:2378:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada455a7480 x1875449616097280/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572394 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788572378.494158:0:2378:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada455a7480 x1875449616097280/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572394 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572378.494171:0:2378:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788572378.494175:0:2378:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788572378.494179:0:2378:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788572378.494748:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788572378.494754:0:2385:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788572378.494763:0:2385:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788572378.494766:0:2385:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788572378.494770:0:2385:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada43d96cb8. 00000400:00000200:3.0:1788572378.494776:0:2385:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada43c8fe58 00000400:00000010:3.0:1788572378.494779:0:2385:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada43c8fe58. 00000100:00000001:3.0:1788572378.494783:0:2385:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572378.494790:0:2385:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada455a7b80 x1875449616097152/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572394 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788572378.494798:0:2385:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada455a7b80 x1875449616097152/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572394 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572378.494804:0:2385:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788572378.494806:0:2385:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788572378.494808:0:2385:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:3.0:1788572378.494812:0:2385:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788572378.494815:0:2385:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788572378.494817:0:2385:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788572378.494820:0:2385:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788572378.494831:0:2385:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada51140460 time=73 v=5 (5 5 5 5) 00000100:00001000:3.0:1788572378.494836:0:2385:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada51140418 time=19 v=5 (1 2 0 0) 00000100:00000001:3.0:1788572378.495025:0:2385:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788572378.495028:0:2385:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788572378.495032:0:2385:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788572378.495036:0:2385:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572378.495040:0:2385:0:(client.c:3110:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788572378.495043:0:2385:0:(client.c:3122:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: skip recheck: last_committed 219043334508 00000100:00000001:3.0:1788572378.495046:0:2385:0:(client.c:3123:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788572378.495048:0:2385:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788572378.495054:0:2385:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada455a7b80 x1875449616097152/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572394 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572378.495131:0:2385:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788572378.495136:0:2385:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788572378.495138:0:2385:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788572378.495143:0:2385:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada455a7b80 x1875449616097152/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572394 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572378.495151:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455a7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616097152:192.168.202.113@tcp:400:kworker.0 00000100:00000001:3.0:1788572378.495157:0:2385:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788572378.495160:0:2385:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788572378.495164:0:2385:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455a7b80 x1875449616097152/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572394 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572378.495171:0:2385:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788572378.495174:0:2385:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788572378.495178:0:2385:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada45574400. 02000000:00000001:3.0:1788572378.495182:0:2385:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788572378.495185:0:2385:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:3.0:1788572378.495187:0:2385:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=3 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:3.0:1788572378.495189:0:2385:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:3.0:1788572378.495192:0:2385:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada50c31600. 02000000:00000001:3.0:1788572378.495196:0:2385:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788572378.495198:0:2385:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788572378.495200:0:2385:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455a7b80. 00000100:00000001:3.0:1788572378.495203:0:2385:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788572378.495205:0:2385:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788572378.495207:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788572378.495216:0:2385:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788572378.495218:0:2385:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788572378.495221:0:2385:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788572378.495224:0:2385:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572378.495274:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788572378.495277:0:2386:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788572378.495282:0:2386:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788572378.495284:0:2386:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788572378.495287:0:2386:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada510c9948. 00000400:00000200:2.0:1788572378.495292:0:2386:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada51269550 00000400:00000010:2.0:1788572378.495293:0:2386:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada51269550. 00000100:00000001:2.0:1788572378.495296:0:2386:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:2.0:1788572378.495301:0:2386:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada455a7480 x1875449616097280/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572394 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788572378.495309:0:2386:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada455a7480 x1875449616097280/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572394 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572378.495315:0:2386:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788572378.495317:0:2386:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572378.495319:0:2386:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:2.0:1788572378.495321:0:2386:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788572378.495322:0:2386:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788572378.495324:0:2386:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572378.495325:0:2386:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788572378.495330:0:2386:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada58b08460 time=140 v=5 (5 5 5 5) 00000100:00001000:2.0:1788572378.495333:0:2386:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada58b08418 time=140 v=5 (1 1 1 1) 00000100:00000001:2.0:1788572378.495335:0:2386:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788572378.495336:0:2386:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572378.495338:0:2386:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788572378.495340:0:2386:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572378.495342:0:2386:0:(client.c:3110:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788572378.495343:0:2386:0:(client.c:3122:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff9ada58b0a000: skip recheck: last_committed 0 00000100:00000001:2.0:1788572378.495345:0:2386:0:(client.c:3123:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788572378.495347:0:2386:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572378.495350:0:2386:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada455a7480 x1875449616097280/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572394 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572378.495356:0:2386:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788572378.495358:0:2386:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788572378.495359:0:2386:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572378.495362:0:2386:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada455a7480 x1875449616097280/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572394 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788572378.495370:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616097280:192.168.202.113@tcp:400:kworker.0 00000100:00000001:2.0:1788572378.495374:0:2386:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572378.495376:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572378.495378:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455a7480 x1875449616097280/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572394 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572378.495384:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572378.495385:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572378.495387:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada46491c00. 02000000:00000001:2.0:1788572378.495390:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572378.495391:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572378.495392:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada58b08000 refcount=3 obd=lustre-MDT0001-mdc-ffff9ada58b0a000 00000020:00000001:2.0:1788572378.495394:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572378.495396:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada50c31500. 02000000:00000001:2.0:1788572378.495398:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572378.495399:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572378.495400:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455a7480. 00000100:00000001:2.0:1788572378.495403:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572378.495404:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572378.495406:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572378.495408:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788572378.495409:0:2386:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788572378.495410:0:2386:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572378.495412:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788572378.496780:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572378.496811:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572378.496817:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572378.496819:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572378.496829:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572378.496846:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572378.497430:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572378.497436:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572378.497443:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572378.497448:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 25 MB=0x6a9b624295f00 00000400:00000200:3.0:1788572378.497654:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572378.497722:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.202.113@tcp of length 224/224 into md 0x358ed [1] + 224 00000800:00000001:3.0:1788572378.497732:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572378.497751:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572378.497755:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572378.497760:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572378.497770:0:2377:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572378.497781:0:2377:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada455a5880 x1875449616097024/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572394 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572378.498651:0:2377:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada455a5880 x1875449616097024/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572394 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572378.499022:0:2377:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572378.500223:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572378.500240:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572378.500900:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788572378.500907:0:2386:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788572378.500918:0:2386:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788572378.500921:0:2386:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788572378.500926:0:2386:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada510c98f0. 00000400:00000200:2.0:1788572378.500937:0:2386:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada51269110 00000400:00000010:2.0:1788572378.500940:0:2386:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada51269110. 00000100:00000001:2.0:1788572378.500946:0:2386:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:2.0:1788572378.500960:0:2386:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada455a5880 x1875449616097024/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572394 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788572378.500973:0:2386:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada455a5880 x1875449616097024/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572394 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572378.500981:0:2386:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788572378.500983:0:2386:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572378.500986:0:2386:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:2.0:1788572378.500989:0:2386:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788572378.500992:0:2386:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788572378.500994:0:2386:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788572378.500997:0:2386:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788572378.501005:0:2386:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada58a62c60 time=140 v=5 (5 5 5 5) 00000100:00001000:2.0:1788572378.501009:0:2386:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada58a62c18 time=20 v=5 (1 1 0 0) 00000100:00000001:2.0:1788572378.501012:0:2386:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788572378.501014:0:2386:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572378.501017:0:2386:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788572378.501019:0:2386:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572378.501022:0:2386:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788572378.501027:0:2386:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada455a5880 x1875449616097024/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572394 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572378.501034:0:2386:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788572378.501038:0:2386:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788572378.501040:0:2386:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788572378.501043:0:2386:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada455a5880 x1875449616097024/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572394 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788572378.501051:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada455a5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449616097024:192.168.202.113@tcp:400:kworker.0 00000100:00000001:2.0:1788572378.501059:0:2386:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572378.501062:0:2386:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572378.501077:0:2386:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada455a5880 x1875449616097024/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572394 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572378.501084:0:2386:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788572378.501087:0:2386:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788572378.501091:0:2386:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada46490400. 02000000:00000001:2.0:1788572378.501095:0:2386:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788572378.501098:0:2386:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788572378.501100:0:2386:0:(genops.c:1223:class_import_put()) import ffff9ada58a62800 refcount=3 obd=MGC192.168.202.113@tcp 00000020:00000001:2.0:1788572378.501102:0:2386:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788572378.501105:0:2386:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada50c31700. 02000000:00000001:2.0:1788572378.501110:0:2386:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788572378.501112:0:2386:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788572378.501114:0:2386:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada455a5880. 00000100:00000001:2.0:1788572378.501120:0:2386:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788572378.501121:0:2386:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572378.501124:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788572378.501133:0:2386:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1788572378.501136:0:2386:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572378.501138:0:2386:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788572378.501140:0:2386:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572379.001287:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572379.001297:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572379.001304:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572379.001305:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572379.001309:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572379.001310:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572379.001395:0:15205:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572379.001398:0:15205:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572379.001401:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572380.023125:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572380.023135:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572380.023141:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572380.023143:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572380.023147:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572380.023148:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572380.023315:0:15204:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572380.023320:0:15204:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572380.023323:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572381.048066:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572381.048076:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572381.048085:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572381.048087:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572381.048090:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572381.048091:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788572381.049553:0:33423:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788572381.049560:0:33423:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788572381.049565:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572382.071248:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572382.071259:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572382.071267:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572382.071268:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572382.071271:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572382.071273:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572382.071658:0:15205:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572382.071663:0:15205:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572382.071666:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788572383.096202:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572383.096212:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572383.096221:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572383.096223:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572383.096227:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572383.096228:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788572383.096839:0:15204:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788572383.096844:0:15204:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788572383.096889:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788572383.612401:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788572383.614132:0:77466:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788572383.614149:0:77466:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada45461500. 00000020:00000040:0.0:1788572383.614157:0:77466:0:(genops.c:1210:class_import_get()) import ffff9ada58a62800 refcount=4 obd=MGC192.168.202.113@tcp 00000100:00000001:0.0:1788572383.614163:0:77466:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788572383.614166:0:77466:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788572383.614172:0:77466:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788572383.614183:0:77466:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada5074ad00. 00000100:00000001:0.0:1788572383.614191:0:77466:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572383.614206:0:77466:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS req@ffff9ada45461500 x1875449616097408/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788572383.614218:0:77466:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788572383.614224:0:77466:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572383.614230:0:77466:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000002269e97f] to pc [ptlrpcd_00_01+1] req@ffff9ada45461500 x1875449616097408/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572383.614269:0:77466:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572383.614272:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788572383.614276:0:77466:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788572383.614279:0:77466:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada45460380. 00000020:00000040:0.0:1788572383.614281:0:77466:0:(genops.c:1210:class_import_get()) import ffff9ada51140000 refcount=4 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000100:00000001:0.0:1788572383.614283:0:77466:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788572383.614284:0:77466:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788572383.614286:0:77466:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788572383.614289:0:77466:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada5074aa00. 00000100:00000001:0.0:1788572383.614291:0:77466:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572383.614296:0:77466:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID req@ffff9ada45460380 x1875449616097536/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788572383.614303:0:77466:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788572383.614305:0:77466:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572383.614308:0:77466:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d6500842] to pc [ptlrpcd_00_00+0] req@ffff9ada45460380 x1875449616097536/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572383.614317:0:77466:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572383.614320:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1788572383.614324:0:77466:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1788572383.614326:0:77466:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ada45460000. 00000020:00000040:0.0:1788572383.614328:0:77466:0:(genops.c:1210:class_import_get()) import ffff9ada58b08000 refcount=4 obd=lustre-MDT0001-mdc-ffff9ada58b0a000 00000100:00000001:0.0:1788572383.614330:0:77466:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1788572383.614331:0:77466:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1788572383.614333:0:77466:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1788572383.614336:0:77466:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ada5074a800. 00000100:00000001:0.0:1788572383.614338:0:77466:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572383.614342:0:77466:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID req@ffff9ada45460000 x1875449616097664/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788572383.614349:0:77466:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788572383.614351:0:77466:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572383.614354:0:77466:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000e159ca87] to pc [ptlrpcd_00_01+1] req@ffff9ada45460000 x1875449616097664/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788572383.614362:0:77466:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788572383.614364:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0000_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:0.0:1788572383.614368:0:77466:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0000_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:0.0:1788572383.614371:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:0.0:1788572383.614374:0:77466:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:0.0:1788572383.614380:0:77466:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (4306) 00000100:00000001:1.0:1788572383.614717:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572383.614722:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788572383.614726:0:2384:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788572383.614734:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada45461500 x1875449616097408/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572383.614742:0:2384:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788572383.614744:0:2384:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572383.614749:0:2384:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788572383.614752:0:2384:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788572383.614755:0:2384:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788572383.614759:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45461500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449616097408:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572383.614763:0:2384:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788572383.614771:0:2384:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788572383.614774:0:2384:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572383.614777:0:2384:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788572383.614784:0:2384:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada607e0a00. 02000000:00000001:1.0:1788572383.614786:0:2384:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788572383.614795:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:1.0:1788572383.614800:0:2384:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada600d16e0. 00000400:00000010:1.0:1788572383.614809:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2110. 00000100:00000200:1.0:1788572383.614818:0:2384:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616097408, portal 25 00000100:00000001:1.0:1788572383.614821:0:2384:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788572383.614822:0:2384:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860988609792 : -111212720941824 : ffff9ada45461500) 00000100:00000040:1.0:1788572383.614829:0:2384:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada45461500 x1875449616097408/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572399 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572383.614837:0:2384:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788572383.614841:0:2384:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:1.0:1788572383.614844:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2bb0. 00000100:00000200:1.0:1788572383.614848:0:2384:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875449616097408, offset 0 00000400:00000200:1.0:1788572383.614898:0:2384:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:1.0:1788572383.614909:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572383.614919:0:2384:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:1.0:1788572383.614926:0:2384:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788572383.614931:0:2384:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:1.0:1788572383.614936:0:2384:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788572383.614948:0:2384:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:1.0:1788572383.614953:0:2384:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22333:22333:256:4294967295] 192.168.202.113@tcp LPNI seq info [22329:22329:8:4294967295] 00000400:00000200:1.0:1788572383.614957:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572383.614968:0:2384:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:1.0:1788572383.614975:0:2384:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:1.0:1788572383.614981:0:2384:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada48e6fc00. 00000800:00000200:1.0:1788572383.614989:0:2384:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:1.0:1788572383.614998:0:2384:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:1.0:1788572383.615003:0:2384:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada48e6fc00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788572383.615024:0:2384:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788572383.615029:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:1.0:1788572383.615032:0:2384:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS new deadline estimation for cpu 2 is 4298979156 00000100:00000001:1.0:1788572383.615039:0:2384:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4298974029 00000100:00000001:1.0:1788572383.615044:0:2384:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS new deadline estimation for cpu 3 is 4298979156 00000100:00000001:1.0:1788572383.615048:0:2384:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.113@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4298974029 00000100:00000001:1.0:1788572383.615051:0:2384:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572383.615054:0:2384:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572383.615057:0:2384:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788572383.615061:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada45460000 x1875449616097664/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572383.615182:0:2384:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788572383.615184:0:2384:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572383.615187:0:2384:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788572383.615188:0:2384:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788572383.615191:0:2384:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788572383.615194:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45460000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616097664:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572383.615198:0:2384:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788572383.615200:0:2384:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788572383.615202:0:2384:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572383.615203:0:2384:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788572383.615207:0:2384:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada607e0c00. 02000000:00000001:1.0:1788572383.615209:0:2384:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788572383.615213:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:1.0:1788572383.615215:0:2384:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada600d1ec8. 00000400:00000010:1.0:1788572383.615219:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2908. 00000100:00000200:1.0:1788572383.615222:0:2384:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616097664, portal 10 00000100:00000001:1.0:1788572383.615224:0:2384:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788572383.615226:0:2384:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860988604416 : -111212720947200 : ffff9ada45460000) 00000100:00000040:1.0:1788572383.615233:0:2384:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada45460000 x1875449616097664/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572399 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572383.615245:0:2384:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788572383.615248:0:2384:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:1.0:1788572383.615251:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d24c8. 00000100:00000200:1.0:1788572383.615253:0:2384:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875449616097664, offset 0 00000400:00000200:1.0:1788572383.615257:0:2384:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:1.0:1788572383.615261:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572383.615265:0:2384:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:1.0:1788572383.615267:0:2384:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788572383.615270:0:2384:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:1.0:1788572383.615272:0:2384:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788572383.615274:0:2384:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:1.0:1788572383.615276:0:2384:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22334:22334:255:4294967295] 192.168.202.113@tcp LPNI seq info [22330:22330:7:4294967295] 00000400:00000200:1.0:1788572383.615280:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572383.615286:0:2384:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:1.0:1788572383.615290:0:2384:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:1.0:1788572383.615293:0:2384:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada48e6f900. 00000800:00000200:1.0:1788572383.615297:0:2384:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:1.0:1788572383.615302:0:2384:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:1.0:1788572383.615305:0:2384:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada48e6f900 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788572383.615308:0:2384:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572383.615312:0:2384:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: using a timeout of 16692 usecs (17 jiffies) 00000400:00000200:1.0:1788572383.615315:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:1.0:1788572383.615329:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298968601 00000100:00000001:1.0:1788572383.615332:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298979156 00000100:00000001:1.0:1788572383.615334:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298968601 00000100:00000001:1.0:1788572383.615336:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298979156 00000100:00000001:1.0:1788572383.615338:0:2384:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572383.615340:0:2384:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572383.615344:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572383.615347:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572383.615355:0:2384:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788572383.615357:0:2384:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1788572383.615360:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572383.615361:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788572383.615363:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788572383.615366:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00000001:1.0:1788572383.615368:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572383.615369:0:2384:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788572383.615371:0:2384:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572383.615373:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572383.615374:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788572383.615375:0:2384:0:(client.c:1814:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1788572383.615380:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ada45460380 x1875449616097536/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572383.615387:0:2384:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788572383.615389:0:2384:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572383.615391:0:2384:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788572383.615392:0:2384:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788572383.615394:0:2384:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788572383.615397:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada45460380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616097536:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572383.615400:0:2384:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788572383.615402:0:2384:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1788572383.615404:0:2384:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572383.615418:0:2384:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1788572383.615422:0:2384:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ada607e0800. 02000000:00000001:1.0:1788572383.615424:0:2384:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788572383.615428:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000010:1.0:1788572383.615431:0:2384:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ada600d1c60. 00000400:00000010:1.0:1788572383.615434:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d2770. 00000100:00000200:1.0:1788572383.615438:0:2384:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875449616097536, portal 10 00000100:00000001:1.0:1788572383.615440:0:2384:0:(client.c:3276:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788572383.615441:0:2384:0:(client.c:3278:ptlrpc_request_addref()) Process leaving (rc=18446632860988605312 : -111212720946304 : ffff9ada45460380) 00000100:00000040:1.0:1788572383.615447:0:2384:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ada45460380 x1875449616097536/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572399 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572383.615455:0:2384:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1788572383.615457:0:2384:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.113@tcp 00000400:00000010:1.0:1788572383.615460:0:2384:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ada600d26e8. 00000100:00000200:1.0:1788572383.615461:0:2384:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875449616097536, offset 0 00000400:00000200:1.0:1788572383.615466:0:2384:0:(lib-move.c:5150:LNetPut()) LNetPut -> 12345-192.168.202.113@tcp 00000400:00000200:1.0:1788572383.615470:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572383.615474:0:2384:0:(lib-move.c:2806:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.113@tcp local destination 00000400:00000200:1.0:1788572383.615476:0:2384:0:(lib-move.c:2437:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788572383.615478:0:2384:0:(lib-move.c:1618:lnet_get_best_ni()) selected best_ni 192.168.202.13@tcp 00000400:00000200:1.0:1788572383.615481:0:2384:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.13@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788572383.615483:0:2384:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.113@tcp 00000400:00000200:1.0:1788572383.615486:0:2384:0:(lib-move.c:1738:lnet_handle_send()) 192.168.202.13@tcp NI seq info: [22335:22335:254:4294967295] 192.168.202.113@tcp LPNI seq info [22331:22331:6:4294967295] 00000400:00000200:1.0:1788572383.615489:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:1.0:1788572383.615495:0:2384:0:(lib-move.c:1854:lnet_handle_send()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp:) -> 192.168.202.113@tcp(192.168.202.113@tcp:192.168.202.113@tcp) : PUT try# 0 00000800:00000200:1.0:1788572383.615498:0:2384:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.113@tcp 00000800:00000010:1.0:1788572383.615504:0:2384:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ada48e6ff00. 00000800:00000200:1.0:1788572383.615508:0:2384:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ada50f7a540] -> 12345-192.168.202.113@tcp (5) 00000800:00000200:1.0:1788572383.615512:0:2384:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.113@tcp ip 192.168.202.113:988 00000800:00000200:1.0:1788572383.615515:0:2384:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ada48e6ff00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1788572383.615517:0:2384:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572383.615520:0:2384:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: using a timeout of 41258 usecs (42 jiffies) 00000400:00000200:1.0:1788572383.615523:0:2384:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000100:00000001:1.0:1788572383.615525:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 2 is 4298968626 00000100:00000001:1.0:1788572383.615528:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS request active for cpu 2, keep current deadline 4298979156 00000100:00000001:1.0:1788572383.615530:0:2384:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS new deadline estimation for cpu 3 is 4298968626 00000100:00000001:1.0:1788572383.615532:0:2384:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ada58b0a000: PM QoS request active for cpu 3, keep current deadline 4298979156 00000100:00000001:1.0:1788572383.615534:0:2384:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572383.615536:0:2384:0:(client.c:1942:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572383.615539:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572383.615541:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572383.615561:0:2383:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572383.615563:0:2383:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1788572383.615808:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788572383.615814:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada48e6fc00. 00000400:00000200:2.0:1788572383.615823:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572383.615830:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:2.0:1788572383.615837:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2bb0 00000400:00000010:2.0:1788572383.615840:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2bb0. 00000100:00000001:2.0:1788572383.615846:0:2378:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:2.0:1788572383.615891:0:2378:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada45461500 x1875449616097408/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572399 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572383.615903:0:2378:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572383.615908:0:2378:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada45461500 x1875449616097408/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572399 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572383.615915:0:2378:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572383.615916:0:2378:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000200:2.0:1788572383.615932:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788572383.615934:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada48e6f900. 00000400:00000200:2.0:1788572383.615937:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572383.615950:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:2.0:1788572383.615953:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d24c8 00000400:00000010:2.0:1788572383.615956:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d24c8. 00000100:00000001:2.0:1788572383.615959:0:2378:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:2.0:1788572383.615962:0:2378:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada45460000 x1875449616097664/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572399 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572383.615969:0:2378:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572383.615972:0:2378:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada45460000 x1875449616097664/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572399 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572383.615978:0:2378:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572383.615979:0:2378:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000200:2.0:1788572383.616334:0:2378:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1788572383.616337:0:2378:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ada48e6ff00. 00000400:00000200:2.0:1788572383.616342:0:2378:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788572383.616347:0:2378:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000400:00000200:2.0:1788572383.616351:0:2378:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d26e8 00000400:00000010:2.0:1788572383.616353:0:2378:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d26e8. 00000100:00000001:2.0:1788572383.616356:0:2378:0:(events.c:35:request_out_callback()) Process entered 00000100:00000200:2.0:1788572383.616360:0:2378:0:(events.c:40:request_out_callback()) @@@ type 5, status 0 req@ffff9ada45460380 x1875449616097536/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572399 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572383.616368:0:2378:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788572383.616371:0:2378:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ada45460380 x1875449616097536/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572399 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788572383.616377:0:2378:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788572383.616378:0:2378:0:(events.c:69:request_out_callback()) Process leaving 00000800:00000001:3.0:1788572383.619908:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572383.619932:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572383.619948:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572383.619952:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572383.619963:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572383.619973:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572383.619984:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572383.619989:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572383.619995:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572383.620001:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 10 MB=0x6a9b624296180 00000400:00000200:3.0:1788572383.620008:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572383.620014:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.113@tcp of length 224/224 into md 0x35915 [1] + 224 00000800:00000001:3.0:1788572383.620024:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572383.620029:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572383.620033:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572383.620039:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572383.620049:0:2377:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572383.620062:0:2377:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada45460000 x1875449616097664/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572399 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572383.620087:0:2377:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada45460000 x1875449616097664/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572399 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572383.620109:0:2377:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572383.620119:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572383.620123:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572383.620125:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572383.620127:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572383.620130:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572383.620134:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572383.620138:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572383.620141:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572383.620143:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572383.620146:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 10 MB=0x6a9b624296100 00000400:00000200:3.0:1788572383.620149:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572383.620154:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.113@tcp of length 224/224 into md 0x3591d [1] + 224 00000800:00000001:3.0:1788572383.620158:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572383.620161:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572383.620164:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572383.620167:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572383.620170:0:2377:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572383.620175:0:2377:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada45460380 x1875449616097536/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572399 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572383.620189:0:2377:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada45460380 x1875449616097536/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572399 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572383.620195:0:2377:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572383.620199:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572383.620202:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788572383.620204:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572383.620205:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572383.620208:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572383.620211:0:2377:0:(lib-move.c:4645:lnet_parse()) TRACE: 192.168.202.13@tcp(192.168.202.13@tcp) <- 192.168.202.113@tcp : PUT - for me 00000400:00000200:3.0:1788572383.620215:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572383.620218:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572383.620220:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572383.620223:0:2377:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.113@tcp of length 224 into portal 25 MB=0x6a9b624296080 00000400:00000200:3.0:1788572383.620226:0:2377:0:(api-ni.c:1618:lnet_nid4_cpt_hash()) Match nid 192.168.202.113@tcp to cpt 1 00000400:00000200:3.0:1788572383.620230:0:2377:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.202.113@tcp of length 224/224 into md 0x3590d [1] + 224 00000800:00000001:3.0:1788572383.620234:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572383.620241:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788572383.620243:0:2377:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788572383.620246:0:2377:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.13@tcp->192.168.202.113@tcp: PUT: OK 00000100:00000001:3.0:1788572383.620249:0:2377:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:3.0:1788572383.620253:0:2377:0:(events.c:82:reply_in_callback()) @@@ type 2, status 0 req@ffff9ada45461500 x1875449616097408/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572399 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788572383.620263:0:2377:0:(events.c:151:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ada45461500 x1875449616097408/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572399 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788572383.620269:0:2377:0:(events.c:165:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788572383.620273:0:2377:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788572383.620275:0:2377:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572383.620997:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572383.621005:0:2384:0:(client.c:1991:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788572383.621015:0:2384:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788572383.621017:0:2384:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788572383.621021:0:2384:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada600d1c60. 00000400:00000200:1.0:1788572383.621030:0:2384:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2770 00000400:00000010:1.0:1788572383.621034:0:2384:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2770. 00000100:00000001:1.0:1788572383.621038:0:2384:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:1.0:1788572383.621046:0:2384:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada45460380 x1875449616097536/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572399 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788572383.621056:0:2384:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada45460380 x1875449616097536/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572399 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572383.621062:0:2384:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788572383.621064:0:2384:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572383.621083:0:2384:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1788572383.621088:0:2384:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788572383.621090:0:2384:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788572383.621092:0:2384:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572383.621095:0:2384:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788572383.621107:0:2384:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada51140460 time=78 v=5 (5 5 5 5) 00000100:00001000:1.0:1788572383.621112:0:2384:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada51140418 time=24 v=5 (1 2 0 0) 00000100:00000001:1.0:1788572383.621114:0:2384:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788572383.621116:0:2384:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572383.621120:0:2384:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788572383.621124:0:2384:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572383.621128:0:2384:0:(client.c:3110:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788572383.621130:0:2384:0:(client.c:3122:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: skip recheck: last_committed 219043334508 00000100:00000001:1.0:1788572383.621133:0:2384:0:(client.c:3123:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788572383.621135:0:2384:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572383.621140:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada45460380 x1875449616097536/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572399 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572383.621154:0:2384:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572383.621158:0:2384:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788572383.621159:0:2384:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788572383.621163:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada45460380 x1875449616097536/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572399 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572383.621171:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45460380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616097536:192.168.202.113@tcp:400:kworker.0 00000400:00000001:1.0:1788572383.621178:0:2384:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788572383.621180:0:2384:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788572383.621182:0:2384:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada600d16e0. 00000400:00000200:1.0:1788572383.621184:0:2384:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2110 00000400:00000010:1.0:1788572383.621186:0:2384:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2110. 00000100:00000001:1.0:1788572383.621188:0:2384:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:1.0:1788572383.621192:0:2384:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada45461500 x1875449616097408/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572399 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788572383.621199:0:2384:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada45461500 x1875449616097408/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572399 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572383.621205:0:2384:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788572383.621206:0:2384:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572383.621208:0:2384:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1788572383.621209:0:2384:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788572383.621210:0:2384:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788572383.621212:0:2384:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572383.621214:0:2384:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788572383.621217:0:2384:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada58a62c60 time=145 v=5 (5 5 5 5) 00000100:00001000:1.0:1788572383.621220:0:2384:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada58a62c18 time=25 v=5 (1 1 0 0) 00000100:00000001:1.0:1788572383.621222:0:2384:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788572383.621224:0:2384:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572383.621225:0:2384:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788572383.621227:0:2384:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572383.621229:0:2384:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572383.621232:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada45461500 x1875449616097408/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572399 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572383.621239:0:2384:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572383.621241:0:2384:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788572383.621242:0:2384:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788572383.621246:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada45461500 x1875449616097408/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572399 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572383.621253:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45461500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449616097408:192.168.202.113@tcp:400:kworker.0 00000400:00000001:1.0:1788572383.621259:0:2384:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788572383.621260:0:2384:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788572383.621262:0:2384:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ada600d1ec8. 00000400:00000200:1.0:1788572383.621264:0:2384:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ada600d2908 00000400:00000010:1.0:1788572383.621266:0:2384:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9ada600d2908. 00000100:00000001:1.0:1788572383.621268:0:2384:0:(events.c:80:reply_in_callback()) Process entered 00000100:00000200:1.0:1788572383.621272:0:2384:0:(events.c:82:reply_in_callback()) @@@ type 6, status 0 req@ffff9ada45460000 x1875449616097664/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572399 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788572383.621280:0:2384:0:(events.c:104:reply_in_callback()) @@@ unlink req@ffff9ada45460000 x1875449616097664/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572399 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572383.621287:0:2384:0:(events.c:165:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788572383.621288:0:2384:0:(client.c:3002:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572383.621290:0:2384:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1788572383.621291:0:2384:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788572383.621292:0:2384:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788572383.621294:0:2384:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788572383.621296:0:2384:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788572383.621300:0:2384:0:(import.c:2195:obd_at_measure()) add 5 to ffff9ada58b08460 time=145 v=5 (5 5 5 5) 00000100:00001000:1.0:1788572383.621303:0:2384:0:(import.c:2195:obd_at_measure()) add 1 to ffff9ada58b08418 time=145 v=5 (1 1 1 1) 00000100:00000001:1.0:1788572383.621305:0:2384:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788572383.621306:0:2384:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788572383.621308:0:2384:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788572383.621310:0:2384:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572383.621312:0:2384:0:(client.c:3110:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788572383.621314:0:2384:0:(client.c:3122:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff9ada58b0a000: skip recheck: last_committed 0 00000100:00000001:1.0:1788572383.621316:0:2384:0:(client.c:3123:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788572383.621317:0:2384:0:(client.c:1798:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788572383.621321:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ada45460000 x1875449616097664/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572399 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572383.621327:0:2384:0:(client.c:2355:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788572383.621329:0:2384:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788572383.621331:0:2384:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788572383.621334:0:2384:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ada45460000 x1875449616097664/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572399 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572383.621341:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada45460000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616097664:192.168.202.113@tcp:400:kworker.0 00000100:00000001:1.0:1788572383.621345:0:2384:0:(client.c:2466:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572383.621348:0:2384:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788572383.621352:0:2384:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45460380 x1875449616097536/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572399 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572383.621359:0:2384:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788572383.621362:0:2384:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788572383.621366:0:2384:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada607e0800. 02000000:00000001:1.0:1788572383.621371:0:2384:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788572383.621374:0:2384:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788572383.621375:0:2384:0:(genops.c:1223:class_import_put()) import ffff9ada51140000 refcount=3 obd=lustre-MDT0000-mdc-ffff9ada58b0a000 00000020:00000001:1.0:1788572383.621378:0:2384:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1788572383.621381:0:2384:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada5074aa00. 02000000:00000001:1.0:1788572383.621386:0:2384:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788572383.621388:0:2384:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788572383.621390:0:2384:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45460380. 00000100:00000001:1.0:1788572383.621394:0:2384:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788572383.621395:0:2384:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572383.621397:0:2384:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788572383.621401:0:2384:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45461500 x1875449616097408/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572399 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572383.621406:0:2384:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788572383.621408:0:2384:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788572383.621410:0:2384:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada607e0a00. 02000000:00000001:1.0:1788572383.621411:0:2384:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788572383.621412:0:2384:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788572383.621414:0:2384:0:(genops.c:1223:class_import_put()) import ffff9ada58a62800 refcount=3 obd=MGC192.168.202.113@tcp 00000020:00000001:1.0:1788572383.621416:0:2384:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1788572383.621417:0:2384:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada5074ad00. 02000000:00000001:1.0:1788572383.621419:0:2384:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788572383.621420:0:2384:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788572383.621421:0:2384:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45461500. 00000100:00000001:1.0:1788572383.621423:0:2384:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788572383.621424:0:2384:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572383.621426:0:2384:0:(client.c:2868:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788572383.621428:0:2384:0:(client.c:2876:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ada45460000 x1875449616097664/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572399 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788572383.621434:0:2384:0:(client.c:2794:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788572383.621436:0:2384:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788572383.621437:0:2384:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ada607e0c00. 02000000:00000001:1.0:1788572383.621438:0:2384:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788572383.621440:0:2384:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788572383.621441:0:2384:0:(genops.c:1223:class_import_put()) import ffff9ada58b08000 refcount=3 obd=lustre-MDT0001-mdc-ffff9ada58b0a000 00000020:00000001:1.0:1788572383.621443:0:2384:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1788572383.621444:0:2384:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ada5074a800. 02000000:00000001:1.0:1788572383.621446:0:2384:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788572383.621447:0:2384:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788572383.621449:0:2384:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ada45460000. 00000100:00000001:1.0:1788572383.621450:0:2384:0:(client.c:2850:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788572383.621451:0:2384:0:(client.c:2909:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572383.621454:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788572383.621464:0:2384:0:(client.c:2640:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788572383.621466:0:2384:0:(client.c:2668:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788572383.621469:0:2384:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788572383.621472:0:2384:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572384.119034:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572384.119044:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572384.119052:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572384.119053:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788572384.119057:0:365:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788572384.119058:0:365:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788572384.119215:0:33423:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788572384.119219:0:33423:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788572384.119222:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000001:2.0:1788572384.459097:0:89029:0:(debug.c:305:cfs_str2mask()) Process entered 00010000:00010000:0.0:1788572385.150802:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572386.167403:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572387.211979:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572388.218545:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788572388.733697:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449616097792:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572388.733808:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616098048:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572388.733846:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1788572388.734075:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616097920:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572388.740725:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616097920:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572388.740882:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616098048:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572388.745681:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449616097792:192.168.202.113@tcp:400:kworker.0 00010000:00010000:1.0:1788572389.241083:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572396.408327:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572397.431282:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:3.0:1788572398.065304:0:88093:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:3.0:1788572398.065342:0:88093:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1788572398.065426:0:88093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 88093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572398.065438:0:88093:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 88093 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572398.065449:0:88093:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 88093 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572398.065491:0:88093:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job bash:35eb2d60-2de0-4b00-9815-d5153edd0057:88093:1875449616252032:192.168.202.113@tcp:101:bash.0 00000100:00100000:3.0:1788572398.065601:0:88093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2480 going to sleep for 11 seconds 00000100:00100000:0.0:1788572398.068346:0:88093:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job bash:35eb2d60-2de0-4b00-9815-d5153edd0057:88093:1875449616252032:192.168.202.113@tcp:101:bash.0 00010000:00010000:0.0:1788572398.068378:0:88093:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572398.068391:0:88093:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572398.068404:0:88093:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572398.068414:0:88093:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572398.068428:0:88093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572398.068436:0:88093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572398.068443:0:88093:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788572398.068467:0:88093:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada604e1f80 x1875449616252032/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788572414 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788572398.068482:0:88093:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572398.068498:0:88093:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1788572398.068505:0:88093:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1788572398.068543:0:88093:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:0.0:1788572398.068550:0:88093:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b245 00000080:00010000:0.0:1788572398.068556:0:88093:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572398.068561:0:88093:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b245 from it ffffab0ec1c2fbb0 00010000:00010000:0.0:1788572398.068564:0:88093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572398.068574:0:88093:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572398.068608:0:88093:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:0.0:1788572398.068625:0:88093:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 304000 00000080:00200008:0.0:1788572398.068658:0:88093:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000002:00010000:0.0:1788572398.068688:0:88093:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: readdir flags 0 00010000:00010000:0.0:1788572398.068710:0:88093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788572398.068724:0:88093:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000100:00100000:0.0:1788572398.070009:0:88093:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job bash:35eb2d60-2de0-4b00-9815-d5153edd0057:88093:1875449616252160:192.168.202.113@tcp:37:bash.0 00000100:00100000:0.0:1788572398.070128:0:88093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60703cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788572398.095632:0:88093:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job bash:35eb2d60-2de0-4b00-9815-d5153edd0057:88093:1875449616252160:192.168.202.113@tcp:37:bash.0 00010000:00010000:1.0:1788572398.096545:0:88093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1788572398.096574:0:88093:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1788572398.096616:0:88093:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:1.0:1788572398.096645:0:88093:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1788572398.096666:0:88093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1788572398.096681:0:88093:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1788572398.096689:0:88093:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1788572398.096697:0:88093:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:1.0:1788572398.096700:0:88093:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b245 00000080:00010000:1.0:1788572398.096706:0:88093:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b245 from it ffffab0ec1c2fcd8 00010000:00010000:1.0:1788572398.096710:0:88093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1788572398.096718:0:88093:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00000080:00200000:1.0:1788572398.098566:0:88093:0:(dir.c:332:ll_iterate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) pos/size0/69632 32bit_api 0 00000080:00200000:1.0:1788572398.098667:0:88093:0:(dir.c:332:ll_iterate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) pos/size18446744073709551614/69632 32bit_api 0 00000080:00200008:1.0:1788572398.098691:0:88093:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 304000 00000080:00200000:3.0:1788572398.126597:0:89280:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:3.0:1788572398.126646:0:89280:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1788572398.126673:0:89280:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00010000:00010000:3.0:1788572398.126693:0:89280:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00000080:00200000:3.0:1788572398.126702:0:89280:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1788572398.126821:0:89280:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:3.0:1788572398.126830:0:89280:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b245 00000080:00010000:3.0:1788572398.126841:0:89280:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b245 from it ffffab0ec0d7fb58 00010000:00010000:3.0:1788572398.128179:0:89280:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 1 00010000:00010000:3.0:1788572398.128199:0:89280:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00200000:3.0:1788572398.128224:0:89280:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:3.0:1788572398.128341:0:89280:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f42.replay-single-*, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), flags=8 00000080:00200000:3.0:1788572398.128346:0:89280:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f42.replay-single-*, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=getattr 00000002:00010000:3.0:1788572398.128375:0:89280:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: f42.replay-single-*,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 01 00010000:00010000:3.0:1788572398.128442:0:89280:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d4e00/0xef30c340bfb4b24c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 89280 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572398.128452:0:89280:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d4e00/0xef30c340bfb4b24c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 89280 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572398.128465:0:89280:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d4e00/0xef30c340bfb4b24c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 89280 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788572398.128497:0:89280:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job checkstat:35eb2d60-2de0-4b00-9815-d5153edd0057:89280:1875449616252288:192.168.202.113@tcp:101:checkstat.0 00000100:00100000:3.0:1788572398.128796:0:89280:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a26c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788572398.133294:0:89280:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job checkstat:35eb2d60-2de0-4b00-9815-d5153edd0057:89280:1875449616252288:192.168.202.113@tcp:101:checkstat.0 00010000:00010000:3.0:1788572398.133329:0:89280:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d4e00/0xef30c340bfb4b24c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 89280 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572398.133347:0:89280:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d4e00/0xef30c340bfb4b24c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 89280 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572398.133357:0:89280:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d4e00/0xef30c340bfb4b24c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 89280 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572398.133365:0:89280:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d4e00/0xef30c340bfb4b24c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 89280 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572398.133382:0:89280:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d4e00/0xef30c340bfb4b24c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 89280 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572398.133388:0:89280:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9ada606d4e00 already unused, calling callback (ffffffffc1966630) 00010000:00010000:3.0:1788572398.133393:0:89280:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d4e00/0xef30c340bfb4b24c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 89280 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572398.133403:0:89280:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d4e00/0xef30c340bfb4b24c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 89280 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572398.133411:0:89280:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d4e00/0xef30c340bfb4b24c lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 89280 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572398.133420:0:89280:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1788572398.133430:0:89280:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d4e00/0xef30c340bfb4b24c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 89280 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788572398.133437:0:89280:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d4e00/0xef30c340bfb4b24c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 89280 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1788572398.133464:0:89280:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0x7, status=0 req@ffff9ada604e0a80 x1875449616252288/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 592/584 e 0 to 0 dl 1788572414 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'checkstat.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1788572398.133698:0:89280:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:3.0:1788572398.140671:0:89280:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:3.0:1788572398.140954:0:89280:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00010000:3.0:1788572398.140970:0:89280:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b245 from it ffffab0ec0d7f930 00010000:00010000:3.0:1788572398.140979:0:89280:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:3.0:1788572398.140988:0:89280:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:0.0:1788572398.457552:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788572398.969894:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616252544:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572398.970019:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1788572398.970027:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449616252416:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572398.970061:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616252672:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572398.988777:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449616252416:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572398.988973:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616252672:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572398.988991:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616252544:192.168.202.113@tcp:400:kworker.0 00010000:00010000:2.0:1788572399.482894:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572400.503469:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572401.528042:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572402.552289:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572403.578468:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788572404.090419:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449616252800:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572404.090703:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616252928:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572404.093207:0:2386:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 219043334908 gen 9 00000100:00100000:3.0:1788572404.093385:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616252928:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572404.093670:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616253056:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572404.103030:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449616252800:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572404.103063:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616253056:192.168.202.113@tcp:400:kworker.0 00010000:00010000:1.0:1788572404.602199:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572405.623267:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572406.649077:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1788572406.685515:0:89845:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == replay-single test 43: mds osc import failure during recovery; don't LBUG ========================================================== 21:40:04 (1788572404) 00010000:00010000:1.0:1788572407.672812:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:0.0:1788572408.451335:0:89897:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1788572408.451352:0:89897:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572408.451381:0:89897:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572408.451404:0:89897:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:0.0:1788572408.451421:0:89897:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:0.0:1788572408.451441:0:89897:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572408.451450:0:89897:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572408.451455:0:89897:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b245 00000080:00010000:0.0:1788572408.451463:0:89897:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b245 from it ffffab0ec1c87cb8 00010000:00010000:0.0:1788572408.451468:0:89897:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:0.0:1788572408.451478:0:89897:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000080:00200008:0.0:1788572408.452577:0:89897:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1788572408.452583:0:89897:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572408.452598:0:89897:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572408.452610:0:89897:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:0.0:1788572408.452620:0:89897:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000080:00200000:0.0:1788572408.452626:0:89897:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572408.452631:0:89897:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572408.452634:0:89897:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b245 00000080:00010000:0.0:1788572408.452639:0:89897:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b245 from it ffffab0ec1c87cb8 00010000:00010000:0.0:1788572408.452643:0:89897:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:0.0:1788572408.452650:0:89897:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00000080:00200000:0.0:1788572408.453453:0:89897:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572408.453468:0:89897:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572408.453478:0:89897:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00010000:00010000:0.0:1788572408.453489:0:89897:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00000080:00200000:0.0:1788572408.453496:0:89897:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572408.453500:0:89897:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572408.453502:0:89897:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b245 00000080:00010000:0.0:1788572408.453507:0:89897:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b245 from it ffffab0ec1c87bb0 00010000:00010000:0.0:1788572408.453510:0:89897:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00010000:00010000:0.0:1788572408.453518:0:89897:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00000080:00200000:0.0:1788572408.453529:0:89897:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:0.0:1788572408.453538:0:89897:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200080:0.0:1788572408.453641:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:0.0:1788572408.453657:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00800000:00000004:0.0:1788572408.453664:0:89897:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: age 4202, max_age 4325 00000100:00100000:0.0:1788572408.460285:0:89897:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:89897:1875449616253184:192.168.202.113@tcp:41:lfs.0 00000100:00100000:0.0:1788572408.460413:0:89897:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada604fe6c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572408.463091:0:89897:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:89897:1875449616253184:192.168.202.113@tcp:41:lfs.0 00800000:00000004:0.0:1788572408.463119:0:89897:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: update ffff9ada4481d198 cache blocks 317301/353529 objects 1023687/1024000 00000080:00200080:0.0:1788572408.463195:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:0.0:1788572408.463212:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00800000:00000004:0.0:1788572408.463220:0:89897:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: age 4202, max_age 4325 00000100:00100000:0.0:1788572408.463250:0:89897:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:89897:1875449616253312:192.168.202.113@tcp:41:lfs.0 00000100:00100000:0.0:1788572408.463306:0:89897:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada604fe6c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572408.469297:0:89897:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:89897:1875449616253312:192.168.202.113@tcp:41:lfs.0 00800000:00000004:0.0:1788572408.469327:0:89897:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: update ffff9ada44819e18 cache blocks 317392/353529 objects 1023736/1024000 00000080:00200080:0.0:1788572408.469398:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:0.0:1788572408.469414:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.469422:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:0.0:1788572408.469426:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.469441:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:0.0:1788572408.469446:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.469451:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:0.0:1788572408.469455:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.469459:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:0.0:1788572408.469463:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.469467:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:0.0:1788572408.469471:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.469475:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:0.0:1788572408.469478:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.469481:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:0.0:1788572408.469485:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.469488:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:0.0:1788572408.469492:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.469496:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:0.0:1788572408.469499:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.469503:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:0.0:1788572408.469506:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.469510:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:0.0:1788572408.469513:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.469517:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:0.0:1788572408.469520:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.469524:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:0.0:1788572408.469528:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.469531:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:0.0:1788572408.469534:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.469538:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:0.0:1788572408.469541:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.469544:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:0.0:1788572408.469548:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.469552:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:0.0:1788572408.469555:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.469559:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:0.0:1788572408.469563:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.469566:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:0.0:1788572408.469570:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.469573:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:0.0:1788572408.469576:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.469580:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:0.0:1788572408.469583:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.469586:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:0.0:1788572408.469589:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.469593:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:0.0:1788572408.469596:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.469599:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:0.0:1788572408.469602:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.469605:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:0.0:1788572408.469608:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.469611:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:0.0:1788572408.469614:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.469617:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:0.0:1788572408.469620:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.469624:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:0.0:1788572408.469626:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.469630:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:0.0:1788572408.469634:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.469638:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:0.0:1788572408.469642:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.469645:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:0.0:1788572408.469649:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.469652:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:0.0:1788572408.469656:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.469660:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:0.0:1788572408.469664:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.469668:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:0.0:1788572408.469671:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.469676:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:0.0:1788572408.469691:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.469696:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:0.0:1788572408.469699:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.469703:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:0.0:1788572408.469706:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:1.0:1788572408.475554:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:1.0:1788572408.475585:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648c00 uarg=0000000000000000 00000080:00200080:1.0:1788572408.475606:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:1.0:1788572408.475612:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648c00 uarg=0000000000000000 00000080:00200080:1.0:1788572408.475616:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:1.0:1788572408.475619:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648c00 uarg=0000000000000000 00000080:00200080:1.0:1788572408.475623:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:1.0:1788572408.475626:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648c00 uarg=0000000000000000 00000080:00200080:1.0:1788572408.475629:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:1.0:1788572408.475633:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648c00 uarg=0000000000000000 00000080:00200080:1.0:1788572408.475636:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:1.0:1788572408.475639:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648c00 uarg=0000000000000000 00000080:00200080:1.0:1788572408.475642:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:1.0:1788572408.475645:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648c00 uarg=0000000000000000 00000080:00200080:1.0:1788572408.475649:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:1.0:1788572408.475652:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648c00 uarg=0000000000000000 00000080:00200080:1.0:1788572408.475656:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:1.0:1788572408.475659:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648c00 uarg=0000000000000000 00000080:00200080:1.0:1788572408.475662:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:1.0:1788572408.475665:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648c00 uarg=0000000000000000 00000080:00200080:1.0:1788572408.475668:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:1.0:1788572408.475672:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648c00 uarg=0000000000000000 00000080:00200080:1.0:1788572408.475675:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:1.0:1788572408.475690:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648c00 uarg=0000000000000000 00000080:00200080:1.0:1788572408.475695:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:1.0:1788572408.475699:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648c00 uarg=0000000000000000 00000080:00200080:1.0:1788572408.475703:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:1.0:1788572408.475707:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648c00 uarg=0000000000000000 00000080:00200080:1.0:1788572408.476948:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:1.0:1788572408.476965:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648c00 uarg=0000000000000000 00000080:00200080:1.0:1788572408.476970:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:1.0:1788572408.476974:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648c00 uarg=0000000000000000 00000080:00200080:1.0:1788572408.476978:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:1.0:1788572408.476982:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648c00 uarg=0000000000000000 00000080:00200080:1.0:1788572408.476986:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:1.0:1788572408.476990:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648c00 uarg=0000000000000000 00000080:00200080:1.0:1788572408.476994:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:1.0:1788572408.476998:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648c00 uarg=0000000000000000 00000080:00200080:1.0:1788572408.477002:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:1.0:1788572408.477006:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648c00 uarg=0000000000000000 00000080:00200080:1.0:1788572408.477009:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:1.0:1788572408.477013:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648c00 uarg=0000000000000000 00000080:00200080:1.0:1788572408.477017:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:1.0:1788572408.477020:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648c00 uarg=0000000000000000 00000080:00200080:1.0:1788572408.477024:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:1.0:1788572408.477027:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648c00 uarg=0000000000000000 00000080:00200080:1.0:1788572408.477031:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:1.0:1788572408.477035:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648c00 uarg=0000000000000000 00000080:00200080:1.0:1788572408.477040:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00800000:00000080:1.0:1788572408.477044:0:89897:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648c00 uarg=0000000000000000 00000080:00200080:1.0:1788572408.477048:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:1.0:1788572408.477054:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648c00 uarg=0000000000000000 00020000:00000004:1.0:1788572408.477063:0:89897:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: age 4202, max_age 4325 00000100:00080000:1.0:1788572408.477082:0:89897:0:(import.c:545:import_select_connection()) lustre-OST0000-osc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4253 00000100:00080000:1.0:1788572408.477099:0:89897:0:(import.c:665:import_select_connection()) lustre-OST0000-osc-ffff9ada58b0a000: import ffff9ada48407800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572408.477113:0:89897:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683394 ocd_grant: 8388608, lost: 0. 00000100:00100000:1.0:1788572408.477145:0:89897:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f800 x1875449616253440/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572408.477199:0:89897:0:(client.c:1846:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != CONNECTING) req@ffff9ada4566df80 x1875449616253568/t0(0) o13->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572408.477211:0:89897:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60703600 going to sleep for 0 seconds 00000100:00100000:2.0:1788572408.477308:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616253440:192.168.202.113@tcp:8:lfs.0 00000100:00100000:1.0:1788572408.484226:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-ffff9ada58b0a000: committing for last_committed 8589935763 gen 6 00000100:00080000:1.0:1788572408.484246:0:2382:0:(import.c:1176:ptlrpc_connect_interpret()) lustre-OST0000-osc-ffff9ada58b0a000: connect to target with instance 3 00000100:00080000:1.0:1788572408.484258:0:2382:0:(import.c:1000:ptlrpc_connect_set_flags()) lustre-OST0000-osc-ffff9ada58b0a000: Resetting ns_connect_flags to server flags: 0xa0427af2e3440038 00000100:00080000:1.0:1788572408.484272:0:2382:0:(import.c:1257:ptlrpc_connect_interpret()) connected to replayable target: lustre-OST0000_UUID 00000100:00080000:1.0:1788572408.484277:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada48407800 lustre-OST0000_UUID: changing import state from CONNECTING to FULL 00000080:00000004:1.0:1788572408.484288:0:2382:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0xa0427af2e3440038 -> 0xa0427af2e3440038 00000080:00080000:1.0:1788572408.484293:0:2382:0:(lcommon_misc.c:56:cl_init_ea_size()) updating def/max_easize: 72/65536 00000100:00080000:1.0:1788572408.484309:0:2382:0:(recover.c:191:ptlrpc_wake_delayed()) @@@ waking (set 00000000fc446c44): req@ffff9ada4566df80 x1875449616253568/t0(0) o13->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572408.484338:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616253440:192.168.202.113@tcp:8:lfs.0 00000100:00100000:2.0:1788572408.484576:0:89897:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60703600 going to sleep for 11 seconds 00000100:00100000:1.0:1788572408.495842:0:89897:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:89897:1875449616253568:192.168.202.113@tcp:13:lfs.0 00020000:00000004:1.0:1788572408.496052:0:89897:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: update ffff9ada4354d198 cache blocks 901356/958279 objects 261713/262144 00000080:00200080:2.0:1788572408.497318:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:2.0:1788572408.497343:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00020000:00000004:2.0:1788572408.497353:0:89897:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: age 4202, max_age 4325 00000100:00080000:2.0:1788572408.497370:0:89897:0:(import.c:545:import_select_connection()) lustre-OST0001-osc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4202 00000100:00080000:2.0:1788572408.497385:0:89897:0:(import.c:665:import_select_connection()) lustre-OST0001-osc-ffff9ada58b0a000: import ffff9ada48400800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572408.497397:0:89897:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683394 ocd_grant: 8437760, lost: 0. 00000100:00100000:2.0:1788572408.497423:0:89897:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616253696/t0(0) o8->lustre-OST0001-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788572408.497488:0:89897:0:(client.c:1846:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != CONNECTING) req@ffff9ada60626a00 x1875449616253824/t0(0) o13->lustre-OST0001-osc-ffff9ada58b0a000@192.168.202.113@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788572408.497499:0:89897:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60a77780 going to sleep for 0 seconds 00000100:00100000:3.0:1788572408.497692:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616253696:192.168.202.113@tcp:8:lfs.0 00000100:00100000:3.0:1788572408.513619:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-ffff9ada58b0a000: committing for last_committed 4294967635 gen 7 00000100:00080000:3.0:1788572408.513640:0:2382:0:(import.c:1176:ptlrpc_connect_interpret()) lustre-OST0001-osc-ffff9ada58b0a000: connect to target with instance 1 00000100:00080000:3.0:1788572408.513654:0:2382:0:(import.c:1000:ptlrpc_connect_set_flags()) lustre-OST0001-osc-ffff9ada58b0a000: Resetting ns_connect_flags to server flags: 0xa0427af2e3440038 00000100:00080000:3.0:1788572408.513672:0:2382:0:(import.c:1257:ptlrpc_connect_interpret()) connected to replayable target: lustre-OST0001_UUID 00000100:00080000:3.0:1788572408.513678:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada48400800 lustre-OST0001_UUID: changing import state from CONNECTING to FULL 00000080:00000004:3.0:1788572408.513740:0:2382:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0xa0427af2e3440038 -> 0xa0427af2e3440038 00000080:00080000:3.0:1788572408.513746:0:2382:0:(lcommon_misc.c:56:cl_init_ea_size()) updating def/max_easize: 72/65536 00000100:00080000:3.0:1788572408.513762:0:2382:0:(recover.c:191:ptlrpc_wake_delayed()) @@@ waking (set 00000000c6b0a7ca): req@ffff9ada60626a00 x1875449616253824/t0(0) o13->lustre-OST0001-osc-ffff9ada58b0a000@192.168.202.113@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572408.513797:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616253696:192.168.202.113@tcp:8:lfs.0 00000100:00100000:2.0:1788572408.514152:0:89897:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60a77780 going to sleep for 11 seconds 00000100:00100000:0.0:1788572408.525837:0:89897:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:89897:1875449616253824:192.168.202.113@tcp:13:lfs.0 00020000:00000004:0.0:1788572408.525911:0:89897:0:(obd_class.h:1116:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: update ffff9ada43549e18 cache blocks 901363/958279 objects 261750/262144 00000080:00200080:0.0:1788572408.525983:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526002:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526010:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526015:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526020:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526024:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526028:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526032:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526037:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526041:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526045:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526049:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526053:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526058:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526062:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526066:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526070:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526073:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526077:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526080:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526083:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526087:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526091:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526094:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526098:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526102:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526106:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526110:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526114:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526118:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526122:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526125:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526130:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526135:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526138:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526143:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526146:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526150:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526153:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526157:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526160:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526164:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526167:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526171:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526175:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526179:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526183:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526186:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526190:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526194:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526198:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526202:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526206:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526210:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526214:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526218:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526222:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526226:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526230:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526234:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526238:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526242:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526247:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526251:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526255:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526259:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526263:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526268:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526272:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526276:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526280:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526284:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526288:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526291:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526295:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526299:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526303:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526306:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526310:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526313:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526317:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526321:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526325:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526330:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526334:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526338:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526341:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526345:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526349:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526353:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526357:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526361:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526365:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526368:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526372:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526375:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526379:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526382:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526386:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526389:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526392:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526396:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526399:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526403:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526407:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526410:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526414:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526418:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526422:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526426:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526430:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526444:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526450:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526453:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526457:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526461:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526465:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526468:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526763:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526774:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526780:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526784:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526789:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526792:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572408.526797:0:89897:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdb4426fe0 00020000:00000080:0.0:1788572408.526801:0:89897:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200008:0.0:1788572408.526817:0:89897:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200000:2.0:1788572408.581325:0:89904:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788572408.581357:0:89904:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788572408.581377:0:89904:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:2.0:1788572408.581393:0:89904:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00000080:00200000:2.0:1788572408.581401:0:89904:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788572408.581409:0:89904:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788572408.581413:0:89904:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b245 00000080:00010000:2.0:1788572408.581421:0:89904:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b245 from it ffffab0ec0d7fbd0 00010000:00010000:2.0:1788572408.581425:0:89904:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:2.0:1788572408.581447:0:89904:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00000080:00200000:2.0:1788572408.581464:0:89904:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:2.0:1788572408.581562:0:89904:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=fsa-oleg213-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), flags=1536 00000080:00200000:2.0:1788572408.581566:0:89904:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788572408.581573:0:89904:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788572408.581580:0:89904:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00010000:00010000:2.0:1788572408.581589:0:89904:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00000080:00200000:2.0:1788572408.581595:0:89904:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788572408.581599:0:89904:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788572408.581602:0:89904:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b245 00000080:00010000:2.0:1788572408.581605:0:89904:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b245 from it ffffab0ec0d7fcd0 00010000:00010000:2.0:1788572408.581608:0:89904:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00010000:00010000:2.0:1788572408.581616:0:89904:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00000080:00200000:2.0:1788572408.581624:0:89904:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:2.0:1788572408.581629:0:89904:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788572408.581633:0:89904:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788572408.581638:0:89904:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00010000:00010000:2.0:1788572408.581652:0:89904:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00000080:00200000:2.0:1788572408.581659:0:89904:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788572408.581661:0:89904:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788572408.581663:0:89904:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b245 00000080:00010000:2.0:1788572408.581666:0:89904:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b245 from it ffffab0ec0d7fdb0 00010000:00010000:2.0:1788572408.581669:0:89904:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00010000:00010000:2.0:1788572408.581675:0:89904:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00000080:00200000:2.0:1788572408.581682:0:89904:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:2.0:1788572408.581686:0:89904:0:(namei.c:2062:ll_create_nd()) VFS Op:name=fsa-oleg213-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), flags=33188, excl=1 00000080:00200000:2.0:1788572408.581690:0:89904:0:(namei.c:2017:ll_mknod()) VFS Op:name=fsa-oleg213-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9ada58125b88) mode 100644 dev 0 00010000:00010000:2.0:1788572408.581714:0:89904:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00010000:00010000:2.0:1788572408.581808:0:89904:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00010000:00010000:2.0:1788572408.581815:0:89904:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1788572408.581818:0:89904:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada606d5600/0xef30c340bfb4b245 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b8339cc5 expref: -99 pid: 88093 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00000020:00100000:2.0:1788572408.581840:0:89904:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1788572408.582642:0:89904:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada60625f80 x1875449616254080/t0(0) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/576 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788572408.582676:0:89904:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job mcreate:35eb2d60-2de0-4b00-9815-d5153edd0057:89904:1875449616254080:192.168.202.113@tcp:36:mcreate.0 00000100:00100000:2.0:1788572408.583194:0:89904:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60a77780 going to sleep for 11 seconds 00000100:00100000:3.0:1788572408.596150:0:89904:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job mcreate:35eb2d60-2de0-4b00-9815-d5153edd0057:89904:1875449616254080:192.168.202.113@tcp:36:mcreate.0 00000080:00200000:3.0:1788572408.596206:0:89904:0:(llite_lib.c:3356:ll_read_inode2()) VFS Op:inode=[0x2000182d1:0x966:0x0](ffff9ada4627b248) 00000080:00200000:3.0:1788572408.596253:0:89904:0:(namei.c:2070:ll_create_nd()) VFS Op:name=fsa-oleg213-client.virtnet, unhashed 1 00000080:00200000:2.0:1788572408.627641:0:89907:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788572408.627671:0:89907:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788572408.627741:0:89907:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8400/0xef30c340bfb4b253 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 89907 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572408.627754:0:89907:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8400/0xef30c340bfb4b253 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 89907 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572408.627767:0:89907:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8400/0xef30c340bfb4b253 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 89907 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572408.627795:0:89907:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:89907:1875449616254208:192.168.202.113@tcp:101:rm.0 00000100:00100000:2.0:1788572408.627907:0:89907:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60a77840 going to sleep for 11 seconds 00000100:00100000:1.0:1788572408.638553:0:89907:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:89907:1875449616254208:192.168.202.113@tcp:101:rm.0 00010000:00010000:1.0:1788572408.638584:0:89907:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8400/0xef30c340bfb4b253 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a022 expref: -99 pid: 89907 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572408.638597:0:89907:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8400/0xef30c340bfb4b253 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a022 expref: -99 pid: 89907 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572408.638612:0:89907:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8400/0xef30c340bfb4b253 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a022 expref: -99 pid: 89907 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572408.638624:0:89907:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8400/0xef30c340bfb4b253 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a022 expref: -99 pid: 89907 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572408.638638:0:89907:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8400/0xef30c340bfb4b253 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a022 expref: -99 pid: 89907 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572408.638647:0:89907:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8400/0xef30c340bfb4b253 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a022 expref: -99 pid: 89907 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572408.638656:0:89907:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8400/0xef30c340bfb4b253 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a022 expref: -99 pid: 89907 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1788572408.638668:0:89907:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada60625180 x1875449616254208/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788572424 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1788572408.638683:0:89907:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8400/0xef30c340bfb4b253 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a022 expref: -99 pid: 89907 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572408.638703:0:89907:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1788572408.638711:0:89907:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1788572408.638740:0:89907:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:1.0:1788572408.638747:0:89907:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b253 00000080:00010000:1.0:1788572408.638753:0:89907:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:1.0:1788572408.638757:0:89907:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b253 from it ffffab0ec1cdfb58 00010000:00010000:1.0:1788572408.638761:0:89907:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8400/0xef30c340bfb4b253 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a022 expref: -99 pid: 89907 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572408.638771:0:89907:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8400/0xef30c340bfb4b253 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a022 expref: -99 pid: 89907 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572408.638794:0:89907:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:1.0:1788572408.638813:0:89907:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=fsa-oleg213-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), flags=8 00000080:00200000:1.0:1788572408.638818:0:89907:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=fsa-oleg213-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9ada58125b88), intent=getattr 00000002:00010000:1.0:1788572408.638842:0:89907:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: fsa-oleg213-client.virtnet,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 01 00010000:00010000:1.0:1788572408.638932:0:89907:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607f3000/0xef30c340bfb4b25a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 89907 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572408.638940:0:89907:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607f3000/0xef30c340bfb4b25a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 89907 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572408.638950:0:89907:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607f3000/0xef30c340bfb4b25a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 89907 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572408.638974:0:89907:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:89907:1875449616254336:192.168.202.113@tcp:101:rm.0 00000100:00100000:1.0:1788572408.639050:0:89907:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60703cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788572408.654354:0:89907:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:89907:1875449616254336:192.168.202.113@tcp:101:rm.0 00010000:00010000:1.0:1788572408.654390:0:89907:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607f3000/0xef30c340bfb4b25a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a03e expref: -99 pid: 89907 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572408.654415:0:89907:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607f3000/0xef30c340bfb4b25a lrc: 4/1,0 mode: --/PR res: [0x2000182d1:0x966:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a03e expref: -99 pid: 89907 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572408.654425:0:89907:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607f3000/0xef30c340bfb4b25a lrc: 4/1,0 mode: --/PR res: [0x2000182d1:0x966:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a03e expref: -99 pid: 89907 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572408.654448:0:89907:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607f3000/0xef30c340bfb4b25a lrc: 4/1,0 mode: PR/PR res: [0x2000182d1:0x966:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a03e expref: -99 pid: 89907 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572408.654462:0:89907:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607f3000/0xef30c340bfb4b25a lrc: 4/1,0 mode: PR/PR res: [0x2000182d1:0x966:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a03e expref: -99 pid: 89907 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572408.654478:0:89907:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607f3000/0xef30c340bfb4b25a lrc: 5/2,0 mode: PR/PR res: [0x2000182d1:0x966:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a03e expref: -99 pid: 89907 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572408.654487:0:89907:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607f3000/0xef30c340bfb4b25a lrc: 5/2,0 mode: PR/PR res: [0x2000182d1:0x966:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a03e expref: -99 pid: 89907 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572408.654496:0:89907:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607f3000/0xef30c340bfb4b25a lrc: 4/1,0 mode: PR/PR res: [0x2000182d1:0x966:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a03e expref: -99 pid: 89907 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1788572408.654509:0:89907:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada4566df80 x1875449616254336/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 600/584 e 0 to 0 dl 1788572424 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1788572408.654524:0:89907:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607f3000/0xef30c340bfb4b25a lrc: 3/1,0 mode: PR/PR res: [0x2000182d1:0x966:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a03e expref: -99 pid: 89907 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572408.654537:0:89907:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8590033617/2406 (0 0) 00000080:00200000:1.0:1788572408.654544:0:89907:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1788572408.654564:0:89907:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x2000182d1:0x966:0x0], updating i_size 0 00000080:00200000:1.0:1788572408.654568:0:89907:0:(namei.c:147:ll_iget()) got inode: [0x2000182d1:0x966:0x0](ffff9ada4627b248): rc = 0 00000080:00200000:1.0:1788572408.654586:0:89907:0:(file.c:6843:ll_layout_conf()) [0x2000182d1:0x966:0x0]: layout version change: 4294967294 -> 4294967295 00000080:00010000:1.0:1788572408.654592:0:89907:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x2000182d1:0x966:0x0](ffff9ada4627b248) for lock 0xef30c340bfb4b25a 00000080:00010000:1.0:1788572408.654603:0:89907:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x2000182d1:0x966:0x0](ffff9ada4627b248) 00000080:00010000:1.0:1788572408.654607:0:89907:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b25a from it ffffab0ec1cdfb20 00010000:00010000:1.0:1788572408.654610:0:89907:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607f3000/0xef30c340bfb4b25a lrc: 3/1,0 mode: PR/PR res: [0x2000182d1:0x966:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b833a03e expref: -99 pid: 89907 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572408.654621:0:89907:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607f3000/0xef30c340bfb4b25a lrc: 2/0,0 mode: PR/PR res: [0x2000182d1:0x966:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b833a03e expref: -99 pid: 89907 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:1.0:1788572408.654660:0:89907:0:(file.c:6296:ll_getattr_dentry()) START file fsa-oleg213-client.virtnet:[0x2000182d1:0x966:0x0](ffff9ada4627b248), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1788572408.654668:0:89907:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x2000182d1:0x966:0x0](ffff9ada4627b248),name=fsa-oleg213-client.virtnet 00000002:00010000:1.0:1788572408.654690:0:89907:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x2000182d1:0x966:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1788572408.654700:0:89907:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607f3000/0xef30c340bfb4b25a lrc: 2/1,0 mode: PR/PR res: [0x2000182d1:0x966:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b833a03e expref: -99 pid: 89907 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1788572408.654712:0:89907:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607f3000/0xef30c340bfb4b25a lrc: 2/1,0 mode: PR/PR res: [0x2000182d1:0x966:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b833a03e expref: -99 pid: 89907 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572408.654720:0:89907:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1788572408.654725:0:89907:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x2000182d1:0x966:0x0](ffff9ada4627b248) 00000080:00010000:1.0:1788572408.654728:0:89907:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x2000182d1:0x966:0x0](ffff9ada4627b248) for lock 0xef30c340bfb4b25a 00000080:00010000:1.0:1788572408.654733:0:89907:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b25a from it ffffab0ec1cdfcb8 00010000:00010000:1.0:1788572408.654736:0:89907:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607f3000/0xef30c340bfb4b25a lrc: 3/1,0 mode: PR/PR res: [0x2000182d1:0x966:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xdb7edb52b833a03e expref: -99 pid: 89907 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1788572408.654744:0:89907:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607f3000/0xef30c340bfb4b25a lrc: 2/0,0 mode: PR/PR res: [0x2000182d1:0x966:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xdb7edb52b833a03e expref: -99 pid: 89907 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1788572408.654804:0:89907:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:1.0:1788572408.654811:0:89907:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1788572408.654818:0:89907:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8400/0xef30c340bfb4b253 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a022 expref: -99 pid: 89907 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1788572408.654827:0:89907:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8400/0xef30c340bfb4b253 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a022 expref: -99 pid: 89907 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:1.0:1788572408.654834:0:89907:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1788572408.654838:0:89907:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:1.0:1788572408.654840:0:89907:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b253 00000080:00010000:1.0:1788572408.654844:0:89907:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b253 from it ffffab0ec1cdfc10 00010000:00010000:1.0:1788572408.654848:0:89907:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8400/0xef30c340bfb4b253 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a022 expref: -99 pid: 89907 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:1.0:1788572408.654903:0:89907:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8400/0xef30c340bfb4b253 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a022 expref: -99 pid: 89907 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1788572408.654912:0:89907:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 1 00000080:00200000:1.0:1788572408.654921:0:89907:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=fsa-oleg213-client.virtnet, flags=0 00000080:00200000:1.0:1788572408.654926:0:89907:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:1.0:1788572408.654933:0:89907:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1788572408.654940:0:89907:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8400/0xef30c340bfb4b253 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a022 expref: -99 pid: 89907 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1788572408.654949:0:89907:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8400/0xef30c340bfb4b253 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a022 expref: -99 pid: 89907 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1788572408.654956:0:89907:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1788572408.654960:0:89907:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:1.0:1788572408.654962:0:89907:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b253 00000080:00010000:1.0:1788572408.654965:0:89907:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b253 from it ffffab0ec1cdfd68 00010000:00010000:1.0:1788572408.654968:0:89907:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8400/0xef30c340bfb4b253 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a022 expref: -99 pid: 89907 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1788572408.654975:0:89907:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8400/0xef30c340bfb4b253 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a022 expref: -99 pid: 89907 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1788572408.654984:0:89907:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 3 00000080:00200000:1.0:1788572408.654990:0:89907:0:(namei.c:2351:ll_unlink()) VFS Op:name=fsa-oleg213-client.virtnet, dir=[0x200000007:0x1:0x0](ffff9ada58125b88) 00010000:00010000:1.0:1788572408.655011:0:89907:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8400/0xef30c340bfb4b253 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0xdb7edb52b833a022 expref: -99 pid: 89907 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1788572408.655034:0:89907:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607f3000/0xef30c340bfb4b25a lrc: 2/0,0 mode: PR/PR res: [0x2000182d1:0x966:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0xdb7edb52b833a03e expref: -99 pid: 89907 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:1.0:1788572408.655044:0:89907:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x966:0x0]: losing layout lock 00010000:00010000:0.0:1788572408.656579:0:89907:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607f3000/0xef30c340bfb4b25a lrc: 1/0,0 mode: --/PR res: [0x2000182d1:0x966:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0xdb7edb52b833a03e expref: -99 pid: 89907 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788572408.656598:0:89907:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8400/0xef30c340bfb4b253 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b833a022 expref: -99 pid: 89907 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:0.0:1788572408.656604:0:89907:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:0.0:1788572408.656608:0:89907:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada607f3000/0xef30c340bfb4b25a lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x966:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0xdb7edb52b833a03e expref: -99 pid: 89907 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788572408.656634:0:89907:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8400/0xef30c340bfb4b253 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0xdb7edb52b833a022 expref: -99 pid: 89907 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000020:00100000:0.0:1788572408.656657:0:89907:0:(genops.c:2428:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff9ada58b0a000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1788572408.656671:0:89907:0:(client.c:820:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff9ada4557d880 x1875449616254592/t0(0) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 512/624 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788572408.656699:0:89907:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:89907:1875449616254592:192.168.202.113@tcp:36:rm.0 00000100:00100000:0.0:1788572408.656918:0:89907:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada604fe6c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572408.692551:0:89907:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job rm:35eb2d60-2de0-4b00-9815-d5153edd0057:89907:1875449616254592:192.168.202.113@tcp:36:rm.0 00000080:00200000:0.0:1788572408.692622:0:89907:0:(vvp_io.c:1826:vvp_io_init()) [0x2000182d1:0x966:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:0.0:1788572408.692634:0:89907:0:(vvp_io.c:304:vvp_io_fini()) [0x2000182d1:0x966:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1788572408.692644:0:89907:0:(llite_lib.c:2066:ll_clear_inode()) VFS Op:inode=[0x2000182d1:0x966:0x0](ffff9ada4627b248) 00010000:00010000:0.0:1788572408.695596:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788572409.209241:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449616254720:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572409.237428:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449616254720:192.168.202.113@tcp:400:kworker.0 00010000:00010000:2.0:1788572409.721392:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572410.744567:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572411.774054:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572412.791171:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572413.819078:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572414.330290:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449616254848:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572414.330464:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616255104:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572414.330498:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616255360:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572414.330528:0:2383:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1788572414.330535:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616254976:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572414.330571:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616255232:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572414.338151:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449616254848:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572414.339278:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616254976:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572414.339295:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616255104:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572414.341852:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616255232:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572414.341904:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616255360:192.168.202.113@tcp:400:kworker.0 00010000:00010000:3.0:1788572414.839147:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572415.864721:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1788572416.616105:0:89974:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: local REPLAY BARRIER on lustre-MDT0000 00010000:00010000:1.0:1788572416.889664:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572417.918965:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572418.935534:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572419.447535:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449616255488:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572419.447640:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616255744:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572419.447682:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616256000:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572419.447712:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1788572419.447718:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616255616:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572419.447740:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616255872:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572419.457072:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616255616:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572419.457093:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616255872:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572419.457105:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449616255488:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572419.457117:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616255744:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572419.457128:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616256000:192.168.202.113@tcp:400:kworker.0 00010000:00010000:1.0:1788572419.959475:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572420.986123:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572422.007070:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572423.031226:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572424.055152:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572424.570728:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616256256:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572424.570912:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616256512:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572424.570949:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:1.0:1788572424.570955:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449616256128:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572424.570995:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616256384:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572424.571017:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616256640:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572424.576934:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616256512:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572424.581289:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616256640:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572424.586143:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616256384:192.168.202.113@tcp:400:kworker.0 00000100:00080000:1.0:1788572424.586162:0:2384:0:(recover.c:205:ptlrpc_request_handle_notconn()) import lustre-MDT0000-mdc-ffff9ada58b0a000 of lustre-MDT0000_UUID@192.168.202.113@tcp abruptly disconnected: reconnecting 00000100:00080000:1.0:1788572424.586168:0:2384:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:1.0:1788572424.586172:0:2384:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff9ada58b0a000: Connection to lustre-MDT0000 (at 192.168.202.113@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:1.0:1788572424.586181:0:2384:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572424.586186:0:2384:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4127 00000100:00080000:1.0:1788572424.586197:0:2384:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572424.586208:0:2384:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572424.586245:0:2384:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566fb80 x1875449616256768/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788572424.586272:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616256256:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572424.586394:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616256768:192.168.202.113@tcp:38:ptlrpcd_00_01.0 00000100:00080000:3.0:1788572424.590329:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572424.590344:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-19) 00000100:00100000:3.0:1788572424.590350:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616256768:192.168.202.113@tcp:38:ptlrpcd_00_01.0 00010000:00010000:2.0:1788572425.080088:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572426.103296:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572427.134940:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572428.151169:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572429.176393:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788572429.687329:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572429.687337:0:58077:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788572429.687342:0:58077:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572429.687353:0:58077:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4342 00000100:00080000:2.0:1788572429.687366:0:58077:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 5s 00000100:00080000:2.0:1788572429.687374:0:58077:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572429.687385:0:58077:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788572429.687406:0:58077:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616257024/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788572429.687448:0:58077:0:(import.c:65:import_set_state_nolock()) ffff9ada48407800 lustre-OST0000_UUID: changing import state from FULL to CONNECTING 00000100:00080000:2.0:1788572429.687452:0:58077:0:(import.c:2147:ptlrpc_disconnect_and_idle_import()) lustre-OST0000-osc-ffff9ada58b0a000: disconnect after 21s idle 00000100:00080000:2.0:1788572429.687460:0:58077:0:(import.c:65:import_set_state_nolock()) ffff9ada48400800 lustre-OST0001_UUID: changing import state from FULL to CONNECTING 00000100:00080000:2.0:1788572429.687463:0:58077:0:(import.c:2147:ptlrpc_disconnect_and_idle_import()) lustre-OST0001-osc-ffff9ada58b0a000: disconnect after 21s idle 00000100:00100000:2.0:1788572429.687546:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449616256896:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572429.687666:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1788572429.687674:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616257152:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572429.688082:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616257024:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572429.688144:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616257280:192.168.202.113@tcp:9:kworker.0 00000100:00100000:1.0:1788572429.688171:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616257408:192.168.202.113@tcp:9:kworker.0 00000100:00080000:1.0:1788572429.696623:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572429.696643:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-19) 00000100:00100000:1.0:1788572429.696649:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616257024:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572429.697957:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616257152:192.168.202.113@tcp:400:kworker.0 00000100:00080000:1.0:1788572429.705314:0:2382:0:(import.c:2057:ptlrpc_disconnect_idle_interpret()) @@@ inflight=1, refcount=3: rc = 0 req@ffff9ada60627b80 x1875449616257280/t0(0) o9->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572439 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572429.705332:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada48407800 lustre-OST0000_UUID: changing import state from CONNECTING to IDLE 00000100:00100000:1.0:1788572429.705347:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616257280:192.168.202.113@tcp:9:kworker.0 00000100:00080000:1.0:1788572429.705366:0:2382:0:(import.c:2057:ptlrpc_disconnect_idle_interpret()) @@@ inflight=1, refcount=3: rc = 0 req@ffff9ada60627800 x1875449616257408/t0(0) o9->lustre-OST0001-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572439 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572429.705374:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada48400800 lustre-OST0001_UUID: changing import state from CONNECTING to IDLE 00000100:00100000:1.0:1788572429.705387:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616257408:192.168.202.113@tcp:9:kworker.0 00010000:00010000:0.0:1788572430.199429:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572431.223050:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572432.247299:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572433.274276:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572434.295197:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788572434.810313:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572434.810321:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788572434.810325:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572434.810338:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4347 00000100:00080000:2.0:1788572434.810351:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 10s 00000100:00080000:2.0:1788572434.810359:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572434.810382:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788572434.810404:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627800 x1875449616257664/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788572434.810561:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616257664:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572434.811096:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449616257536:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572434.811158:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616257792:192.168.202.113@tcp:400:kworker.0 00000100:00080000:1.0:1788572434.813985:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572434.814004:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-19) 00000100:00100000:1.0:1788572434.814010:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616257664:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572434.814394:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616257792:192.168.202.113@tcp:400:kworker.0 00010000:00010000:0.0:1788572435.323909:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572436.343089:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572437.367041:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572438.391189:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572439.417777:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788572439.927707:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572439.927716:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788572439.927720:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572439.927732:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4352 00000100:00080000:0.0:1788572439.927744:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 15s 00000100:00080000:0.0:1788572439.927752:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788572439.927763:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788572439.927784:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616258048/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788572439.927907:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616258176:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572439.928009:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449616257920:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572439.928407:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616258048:192.168.202.113@tcp:38:kworker.0 00000100:00080000:1.0:1788572439.936442:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572439.936460:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-19) 00000100:00100000:1.0:1788572439.936466:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616258048:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788572439.939686:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616258176:192.168.202.113@tcp:400:kworker.0 00010000:00010000:1.0:1788572440.449230:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000400:1.0:1788572440.951013:0:2384:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788572424/real 1788572424] req@ffff9ada4566df80 x1875449616256128/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 1 dl 1788572440 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572441.010757:0:2384:0:(client.c:2548:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9ada4566df80 x1875449616256128/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 1 dl 1788572440 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572441.010774:0:2384:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from FULL to DISCONN 00000100:02020000:1.0:1788572441.010778:0:2384:0:(import.c:186:ptlrpc_set_import_discon()) MGC192.168.202.113@tcp: Connection to MGS (at 192.168.202.113@tcp) was lost; in progress operations using this service will fail 00000100:00080000:1.0:1788572441.037945:0:2384:0:(import.c:192:ptlrpc_set_import_discon()) import MGS@192.168.202.113@tcp for MGC192.168.202.113@tcp not replayable, auto-deactivating 00000100:00080000:1.0:1788572441.037952:0:2384:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import MGS INVALID 00000100:00100000:1.0:1788572441.037963:0:2384:0:(client.c:3591:ptlrpc_abort_inflight()) @@@ inflight req@ffff9ada4566df80 x1875449616256128/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 1 dl 1788572440 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572441.037981:0:2384:0:(client.c:3591:ptlrpc_abort_inflight()) @@@ inflight req@ffff9ada60627480 x1875449616256896/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572445 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572441.038011:0:2384:0:(client.c:3591:ptlrpc_abort_inflight()) @@@ inflight req@ffff9ada60624e00 x1875449616257536/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572450 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572441.038020:0:2384:0:(client.c:3591:ptlrpc_abort_inflight()) @@@ inflight req@ffff9ada4557d180 x1875449616257920/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572455 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:1.0:1788572441.038030:0:2384:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808001 00000100:00080000:1.0:1788572441.038032:0:2384:0:(pinger.c:193:ptlrpc_pinger_ir_down()) IR down 10000000:01000000:1.0:1788572441.038034:0:2384:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808002 00000100:00080000:1.0:1788572441.038036:0:2384:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00100000:1.0:1788572441.040005:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449616257920:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572441.040018:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449616256128:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572441.040917:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449616257536:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572441.040931:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449616256896:192.168.202.113@tcp:400:kworker.0 00000100:00080000:0.0:1788572441.041810:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572441.041818:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788572441.041832:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572441.041841:0:77466:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4126 00000100:00080000:0.0:1788572441.041945:0:77466:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788572441.041985:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616258304/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788572441.042070:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616258304:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788572441.044930:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572441.044941:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788572441.044945:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572441.044957:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4357 00000100:00080000:0.0:1788572441.044970:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572441.044978:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788572441.044994:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788572441.045029:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616258432/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788572441.045237:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616258432:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572441.046815:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616258560:192.168.202.113@tcp:400:kworker.0 00000100:00080000:3.0:1788572441.054066:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572441.054089:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-19) 00000100:00100000:3.0:1788572441.054095:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616258432:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788572441.059775:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616258560:192.168.202.113@tcp:400:kworker.0 00010000:00010000:0.0:1788572441.466112:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572442.487266:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572443.515818:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572444.536428:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788572445.559090:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1788572445.559127:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616258688:192.168.202.113@tcp:17:kworker.0 00010000:00010000:1.0:1788572445.559176:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572445.575279:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616258688:192.168.202.113@tcp:17:kworker.0 00000100:00080000:3.0:1788572446.076718:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572446.076729:0:85951:0:(pinger.c:247:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1788572446.076732:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572446.076736:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788572446.076753:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572446.076765:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4358 00000100:00080000:3.0:1788572446.076777:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572446.076785:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572446.076798:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572446.076838:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1500 x1875449616258816/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788572446.081282:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616258816:192.168.202.113@tcp:38:kworker.0 00000100:00080000:1.0:1788572446.090231:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572446.090247:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-19) 00000100:00100000:1.0:1788572446.090253:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616258816:192.168.202.113@tcp:38:kworker.0 00010000:00010000:1.0:1788572446.583139:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572447.610845:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572448.639733:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572449.655175:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572450.679939:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788572451.191238:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572451.191251:0:85951:0:(pinger.c:247:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1788572451.191256:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572451.191261:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788572451.191266:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572451.191289:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4363 00000100:00080000:3.0:1788572451.191304:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1788572451.191305:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788572441/real 1788572441] req@ffff9ada4557e300 x1875449616258304/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572451 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572451.191317:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572451.191341:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572451.191359:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616258304/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572451 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572451.191387:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3100 x1875449616258944/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572451.191398:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572451.191491:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572451.191497:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616258304:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788572451.191528:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616259072:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572451.191535:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616258944:192.168.202.113@tcp:38:kworker.0 00000100:00080000:3.0:1788572451.193324:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572451.193334:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788572451.193338:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572451.193348:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4358 00000100:00080000:3.0:1788572451.193358:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 5s 00000100:00080000:3.0:1788572451.193367:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788572451.193404:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3800 x1875449616259200/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788572451.193647:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616259200:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788572451.197786:0:2382:0:(import.c:1176:ptlrpc_connect_interpret()) MGC192.168.202.113@tcp: connect to target with instance 0 10000000:01000000:1.0:1788572451.197812:0:2382:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808005 00000100:02000400:1.0:1788572451.197824:0:2382:0:(import.c:1328:ptlrpc_connect_interpret()) Evicted from MGS (at 192.168.202.113@tcp) after server handle changed from 0xdb7edb52b831c954 to 0xdb7edb52b833a260 00010000:00010000:3.0:1788572451.198915:0:2386:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 0/0 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1788572451.198934:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616259072:192.168.202.113@tcp:400:kworker.0 00010000:00010000:2.0:1788572451.199734:0:33423:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 0 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1788572451.199757:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572451.199759:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572451.199761:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788572451.217684:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to EVICTED 00000100:00080000:1.0:1788572451.217707:0:2382:0:(import.c:1718:ptlrpc_import_recovery_state_machine()) evicted from MGS@192.168.202.113@tcp; invalidating 00000100:00100000:3.0:1788572451.220092:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616259200:192.168.202.113@tcp:250:kworker.0 00010000:00080000:3.0:1788572451.220165:0:2382:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff9ada604e3100 x1875449616258944/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/416 e 0 to 0 dl 1788572476 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572451.220191:0:2382:0:(import.c:1176:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to target with instance 52 00000100:00080000:3.0:1788572451.220216:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to REPLAY 00000100:00080000:3.0:1788572451.220222:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:3.0:1788572451.220225:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 219043334908 gen 9 00000100:00100000:3.0:1788572451.220232:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60625f80 x1875449616254080/t219043334909(219043334909) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572424 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572451.220240:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 219043334908 last 0 00000100:00080000:3.0:1788572451.220253:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada60625f80 x1875449616254080/t219043334909(219043334909) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572424 ref 1 fl New:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572451.220270:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616258944:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572451.220321:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616254080:192.168.202.113@tcp:36:mcreate.0 00000100:00080000:1.0:1788572451.222597:0:90249:0:(import.c:1642:ptlrpc_invalidate_import_thread()) thread invalidate import MGC192.168.202.113@tcp to MGS@192.168.202.113@tcp 00000100:00100000:1.0:1788572451.225737:0:90249:0:(import.c:323:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 10000000:01000000:1.0:1788572451.225752:0:90249:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808003 00010000:00010000:1.0:1788572451.225777:0:90249:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.202.113@tcp lock: ffff9ada4542b400/0xef30c340bfb45a47 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x110e400000000 nid: local remote: 0xdb7edb52b831ca81 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1788572451.225791:0:90249:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.202.113@tcp lock: ffff9ada4542b400/0xef30c340bfb45a47 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x110f400000000 nid: local remote: 0xdb7edb52b831ca81 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1788572451.225798:0:90249:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1788572451.226222:0:90249:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1788572451.226236:0:90249:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.202.113@tcp lock: ffff9ada4542b400/0xef30c340bfb45a47 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x590f400000000 nid: local remote: 0xdb7edb52b831ca81 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572451.226255:0:90249:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.202.113@tcp lock: ffff9ada4542be00/0xef30c340bfb45a55 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0xdb7edb52b831ca8f expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1788572451.226262:0:90249:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.202.113@tcp lock: ffff9ada4542be00/0xef30c340bfb45a55 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0xdb7edb52b831ca8f expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1788572451.226267:0:90249:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:0.0:1788572451.239744:0:90249:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1788572451.239771:0:90249:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.202.113@tcp lock: ffff9ada4542be00/0xef30c340bfb45a55 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0xdb7edb52b831ca8f expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572451.239812:0:90249:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.202.113@tcp lock: ffff9ada607f2000/0xef30c340bfb4a74e lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0xdb7edb52b8334db1 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1788572451.239821:0:90249:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.202.113@tcp lock: ffff9ada607f2000/0xef30c340bfb4a74e lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0xdb7edb52b8334db1 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1788572451.239827:0:90249:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:0.0:1788572451.239835:0:90249:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1788572451.239839:0:90249:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.202.113@tcp lock: ffff9ada607f2000/0xef30c340bfb4a74e lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0xdb7edb52b8334db1 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572451.239890:0:90249:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.202.113@tcp lock: ffff9ada51786000/0xef30c340bfb45a4e lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0xdb7edb52b831ca88 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1788572451.239904:0:90249:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.202.113@tcp lock: ffff9ada51786000/0xef30c340bfb45a4e lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0xdb7edb52b831ca88 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1788572451.239910:0:90249:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:0.0:1788572451.239914:0:90249:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1788572451.239918:0:90249:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.202.113@tcp lock: ffff9ada51786000/0xef30c340bfb45a4e lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0xdb7edb52b831ca88 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1788572451.239954:0:90249:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from EVICTED to RECOVER 00000100:00080000:0.0:1788572451.239961:0:90249:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from RECOVER to FULL 10000000:01000000:0.0:1788572451.239964:0:90249:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808004 00000100:00080000:0.0:1788572451.239968:0:90249:0:(pinger.c:186:ptlrpc_pinger_ir_up()) IR up 00000100:02000000:0.0:1788572451.239977:0:90249:0:(import.c:1787:ptlrpc_import_recovery_state_machine()) MGC192.168.202.113@tcp: Connection restored to 192.168.202.113@tcp (at 192.168.202.113@tcp) 00010000:00010000:1.0:1788572451.704580:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788572452.406396:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada60625f80 x1875449616254080/t219043334909(219043334909) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572467 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572452.406415:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:3.0:1788572452.406419:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 219043334908 gen 9 00000100:00100000:3.0:1788572452.406424:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60625f80 x1875449616254080/t219043334909(219043334909) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572467 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572452.406432:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 219043334908 last 219043334909 00000100:00080000:3.0:1788572452.406441:0:2382:0:(client.c:3554:ptlrpc_replay_req()) @@@ REPLAY req@ffff9ada4557d880 x1875449616254592/t219043334910(219043334910) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 512/456 e 0 to 0 dl 1788572424 ref 1 fl New:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572452.406454:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616254080:192.168.202.113@tcp:36:mcreate.0 00000100:00100000:3.0:1788572452.406479:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616254592:192.168.202.113@tcp:36:rm.0 00000100:00080000:3.0:1788572452.417542:0:2382:0:(client.c:3439:ptlrpc_replay_interpret()) @@@ got reply req@ffff9ada4557d880 x1875449616254592/t219043334910(219043334910) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 512/456 e 0 to 0 dl 1788572468 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572452.417563:0:2382:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:3.0:1788572452.417566:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 219043334908 gen 9 00000100:00100000:3.0:1788572452.417573:0:2382:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff9ada60625f80 x1875449616254080/t219043334909(219043334909) o36->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 528/448 e 0 to 0 dl 1788572467 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'mcreate.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572452.417583:0:2382:0:(recover.c:53:ptlrpc_replay_next()) import ffff9ada51140000 from lustre-MDT0000_UUID committed 219043334908 last 219043334910 00000100:00080000:3.0:1788572452.417588:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from REPLAY to REPLAY_LOCKS 00000100:00100000:1.0:1788572452.424445:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616254592:192.168.202.113@tcp:36:rm.0 00010000:00080000:3.0:1788572452.424555:0:90251:0:(ldlm_request.c:2972:ldlm_lock_replay_thread()) lock replay thread lustre-MDT0000-mdc-ffff9ada58b0a000 to lustre-MDT0000_UUID@192.168.202.113@tcp 00010000:00010000:3.0:1788572452.424561:0:90251:0:(ldlm_request.c:2888:ldlm_cancel_unused_locks_for_replay()) Dropping as many unused locks as possible before replay for namespace lustre-MDT0000-mdc-ffff9ada58b0a000 (3) 00000080:00200000:3.0:1788572452.424606:0:90251:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x643:0x0]: losing layout lock 00010000:00010000:3.0:1788572452.424652:0:90251:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada6035c400/0xef30c340bfb47082 lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b83267ad expref: -99 pid: 87653 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1788572452.424669:0:90251:0:(vvp_object.c:117:vvp_conf_set()) [0x2000182d1:0x643:0x0]: losing layout lock 00010000:00010000:3.0:1788572452.424675:0:90251:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9600/0xef30c340bfb47058 lrc: 0/0,0 mode: --/CR res: [0x2000182d1:0x643:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0xdb7edb52b8326736 expref: -99 pid: 87602 timeout: 0 lvb_type: 3 lru_score: 3 lru_type: 0 00010000:00010000:3.0:1788572452.424699:0:90251:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455e9400/0xef30c340bfb4706d lrc: 0/0,0 mode: --/PR res: [0x2000182d1:0x643:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000000 nid: local remote: 0xdb7edb52b8326775 expref: -99 pid: 87616 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1788572452.424716:0:90251:0:(ldlm_request.c:2901:ldlm_cancel_unused_locks_for_replay()) Canceled 0 unused locks from namespace lustre-MDT0000-mdc-ffff9ada58b0a000 00000100:00080000:3.0:1788572452.425462:0:90251:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from REPLAY_LOCKS to REPLAY_WAIT 00000100:00100000:2.0:1788572452.425774:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616259328:192.168.202.113@tcp:400:ldlm_lock_repla.0 00000100:00100000:3.0:1788572452.439523:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ada58b0a000: committing for last_committed 219043334910 gen 9 00000100:00080000:3.0:1788572452.439568:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from REPLAY_WAIT to RECOVER 00000100:00080000:3.0:1788572452.439584:0:2382:0:(client.c:3248:ptlrpc_resend_req()) @@@ going to resend req@ffff9ada604e3100 x1875449616259328/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572468 ref 1 fl Interpret:RQU/2c0/0 rc 0/0 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572452.439596:0:2382:0:(client.c:3257:ptlrpc_resend_req()) @@@ it has reply, so skip it req@ffff9ada604e3100 x1875449616259328/t0(0) o400->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 0 dl 1788572468 ref 1 fl Interpret:RQU/2c0/0 rc 0/0 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572452.439603:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from RECOVER to FULL 00000100:02000000:2.0:1788572452.439955:0:2382:0:(import.c:1787:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff9ada58b0a000: Connection restored to 192.168.202.113@tcp (at 192.168.202.113@tcp) 00000100:00100000:2.0:1788572452.439963:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616259328:192.168.202.113@tcp:400:ldlm_lock_repla.0 00010000:00010000:0.0:1788572452.727444:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:2.0:1788572452.983665:0:15207:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:2.0:1788572452.983675:0:15207:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-cliir-ffff9ada58b0a000 from 209 10000000:01000000:2.0:1788572452.983680:0:15207:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:2.0:1788572452.983756:0:15207:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.202.113@tcp lock: ffff9ada455a9200/0xef30c340bfb4b261 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572452.983768:0:15207:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.202.113@tcp lock: ffff9ada455a9200/0xef30c340bfb4b261 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572452.983781:0:15207:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.202.113@tcp lock: ffff9ada455a9200/0xef30c340bfb4b261 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572452.983815:0:15207:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449616259456:192.168.202.113@tcp:101:ll_cfg_requeue.0 00000100:00100000:2.0:1788572452.984064:0:15207:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60a77c00 going to sleep for 11 seconds 00000100:00100000:1.0:1788572452.990569:0:15207:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449616259456:192.168.202.113@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1788572452.990609:0:15207:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.202.113@tcp lock: ffff9ada455a9200/0xef30c340bfb4b261 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b833a3ef expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572452.990624:0:15207:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.202.113@tcp lock: ffff9ada455a9200/0xef30c340bfb4b261 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0xdb7edb52b833a3ef expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572452.990745:0:15207:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449616259584:192.168.202.113@tcp:256:ll_cfg_requeue.0 00000100:00100000:1.0:1788572452.990848:0:15207:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60703240 going to sleep for 11 seconds 00000100:00100000:1.0:1788572453.011357:0:15207:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449616259584:192.168.202.113@tcp:256:ll_cfg_requeue.0 10000000:01000000:1.0:1788572453.011486:0:15207:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.202.113@tcp: configuration from log 'lustre-cliir' succeeded (0). 00010000:00010000:1.0:1788572453.011498:0:15207:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.202.113@tcp lock: ffff9ada455a9200/0xef30c340bfb4b261 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b833a3ef expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572453.011511:0:15207:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.202.113@tcp lock: ffff9ada455a9200/0xef30c340bfb4b261 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b833a3ef expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1788572453.011528:0:15207:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:1.0:1788572453.011531:0:15207:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff9ada58b0a000 from 52 10000000:01000000:1.0:1788572453.011534:0:15207:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:1.0:1788572453.011583:0:15207:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.202.113@tcp lock: ffff9ada607f2600/0xef30c340bfb4b268 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572453.011592:0:15207:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.202.113@tcp lock: ffff9ada607f2600/0xef30c340bfb4b268 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572453.011602:0:15207:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.202.113@tcp lock: ffff9ada607f2600/0xef30c340bfb4b268 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572453.011628:0:15207:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449616259712:192.168.202.113@tcp:101:ll_cfg_requeue.0 00000100:00100000:1.0:1788572453.011728:0:15207:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60703240 going to sleep for 11 seconds 00000100:00100000:1.0:1788572453.018381:0:15207:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449616259712:192.168.202.113@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1788572453.018423:0:15207:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.202.113@tcp lock: ffff9ada607f2600/0xef30c340bfb4b268 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b833a3f6 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572453.018438:0:15207:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.202.113@tcp lock: ffff9ada607f2600/0xef30c340bfb4b268 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0xdb7edb52b833a3f6 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572453.018623:0:15207:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449616259840:192.168.202.113@tcp:501:ll_cfg_requeue.0 00000100:00100000:1.0:1788572453.018833:0:15207:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60703240 going to sleep for 11 seconds 00000100:00100000:1.0:1788572453.029445:0:15207:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449616259840:192.168.202.113@tcp:501:ll_cfg_requeue.0 00000100:00100000:1.0:1788572453.029542:0:15207:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449616259968:192.168.202.113@tcp:503:ll_cfg_requeue.0 00000100:00100000:1.0:1788572453.029692:0:15207:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60703240 going to sleep for 11 seconds 00000100:00100000:1.0:1788572453.054031:0:15207:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449616259968:192.168.202.113@tcp:503:ll_cfg_requeue.0 00000040:00080000:2.0:1788572453.057514:0:90253:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 64768 count 51 00000020:01000000:1.0:1788572453.061407:0:15207:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 52-51 (rc=0) 10000000:01000000:1.0:1788572453.061455:0:15207:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.202.113@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:1.0:1788572453.061470:0:15207:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.202.113@tcp lock: ffff9ada607f2600/0xef30c340bfb4b268 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b833a3f6 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572453.061486:0:15207:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.202.113@tcp lock: ffff9ada607f2600/0xef30c340bfb4b268 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b833a3f6 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1788572453.061498:0:15207:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:1.0:1788572453.061502:0:15207:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff9ada58b0a000 from 7 10000000:01000000:1.0:1788572453.061505:0:15207:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:1.0:1788572453.061571:0:15207:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.202.113@tcp lock: ffff9ada607f3600/0xef30c340bfb4b26f lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572453.061581:0:15207:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.202.113@tcp lock: ffff9ada607f3600/0xef30c340bfb4b26f lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572453.061596:0:15207:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.202.113@tcp lock: ffff9ada607f3600/0xef30c340bfb4b26f lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572453.061631:0:15207:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449616260096:192.168.202.113@tcp:101:ll_cfg_requeue.0 00000100:00100000:1.0:1788572453.062233:0:15207:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60703240 going to sleep for 11 seconds 00000100:00100000:1.0:1788572453.074080:0:15207:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449616260096:192.168.202.113@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1788572453.074135:0:15207:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.202.113@tcp lock: ffff9ada607f3600/0xef30c340bfb4b26f lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b833a3fd expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788572453.074159:0:15207:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.202.113@tcp lock: ffff9ada607f3600/0xef30c340bfb4b26f lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0xdb7edb52b833a3fd expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788572453.074270:0:15207:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449616260224:192.168.202.113@tcp:501:ll_cfg_requeue.0 00000100:00100000:1.0:1788572453.074373:0:15207:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60703240 going to sleep for 11 seconds 00000100:00100000:3.0:1788572453.089685:0:15207:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449616260224:192.168.202.113@tcp:501:ll_cfg_requeue.0 00000100:00100000:3.0:1788572453.090205:0:15207:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449616260352:192.168.202.113@tcp:503:ll_cfg_requeue.0 00000100:00100000:3.0:1788572453.090383:0:15207:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60478840 going to sleep for 11 seconds 00000100:00100000:3.0:1788572453.095536:0:15207:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449616260352:192.168.202.113@tcp:503:ll_cfg_requeue.0 00000040:00080000:1.0:1788572453.107380:0:90254:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x2:0x0] index 64768 count 7 00000020:01000000:0.0:1788572453.118300:0:15207:0:(obd_config.c:2145:class_config_parse_llog()) Processed log params gen 7-6 (rc=0) 10000000:01000000:0.0:1788572453.118351:0:15207:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.202.113@tcp: configuration from log 'params' succeeded (0). 00010000:00010000:0.0:1788572453.118368:0:15207:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.202.113@tcp lock: ffff9ada607f3600/0xef30c340bfb4b26f lrc: 3/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b833a3fd expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572453.118383:0:15207:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.202.113@tcp lock: ffff9ada607f3600/0xef30c340bfb4b26f lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b833a3fd expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1788572453.118395:0:15207:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:0.0:1788572453.118399:0:15207:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff9ada42e899c0 from 1 10000000:01000000:0.0:1788572453.118402:0:15207:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:0.0:1788572453.118459:0:15207:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.202.113@tcp lock: ffff9ada4547f200/0xef30c340bfb4b276 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572453.118467:0:15207:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.202.113@tcp lock: ffff9ada4547f200/0xef30c340bfb4b276 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572453.118480:0:15207:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.202.113@tcp lock: ffff9ada4547f200/0xef30c340bfb4b276 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572453.118696:0:15207:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449616260480:192.168.202.113@tcp:101:ll_cfg_requeue.0 00000100:00100000:0.0:1788572453.118826:0:15207:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada604fecc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572453.131661:0:15207:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449616260480:192.168.202.113@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1788572453.131720:0:15207:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.202.113@tcp lock: ffff9ada4547f200/0xef30c340bfb4b276 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b833a404 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572453.131736:0:15207:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.202.113@tcp lock: ffff9ada4547f200/0xef30c340bfb4b276 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0xdb7edb52b833a404 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788572453.131846:0:15207:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449616260608:192.168.202.113@tcp:501:ll_cfg_requeue.0 00000100:00100000:0.0:1788572453.132165:0:15207:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada604fecc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572453.140203:0:15207:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:15207:1875449616260608:192.168.202.113@tcp:501:ll_cfg_requeue.0 10000000:01000000:0.0:1788572453.140266:0:15207:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.202.113@tcp: configuration from log 'lustre-sptlrpc' failed (-2). 00010000:00010000:0.0:1788572453.140280:0:15207:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.202.113@tcp lock: ffff9ada4547f200/0xef30c340bfb4b276 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b833a404 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572453.140292:0:15207:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.202.113@tcp lock: ffff9ada4547f200/0xef30c340bfb4b276 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0xdb7edb52b833a404 expref: -99 pid: 15207 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572453.751152:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572454.780615:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572455.799060:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572456.312569:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449616260736:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572456.312714:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616260992:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572456.312757:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1788572456.312764:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616260864:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572456.317097:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616260864:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572456.317121:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449616260736:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572456.317133:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616260992:192.168.202.113@tcp:400:kworker.0 00010000:00010000:2.0:1788572456.825866:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572457.849058:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:2.0:1788572457.893190:0:90306:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1788572457.893208:0:90306:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788572457.893237:0:90306:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788572457.893323:0:90306:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 90306 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572457.893334:0:90306:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 90306 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788572457.893346:0:90306:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 90306 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788572457.893381:0:90306:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:90306:1875449616261120:192.168.202.113@tcp:101:lfs.0 00000100:00100000:2.0:1788572457.893485:0:90306:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60a77b40 going to sleep for 11 seconds 00000100:00100000:0.0:1788572457.905763:0:90306:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:90306:1875449616261120:192.168.202.113@tcp:101:lfs.0 00010000:00010000:0.0:1788572457.905800:0:90306:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572457.905816:0:90306:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572457.905830:0:90306:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572457.905843:0:90306:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572457.905902:0:90306:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572457.905912:0:90306:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572457.905929:0:90306:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1788572457.905943:0:90306:0:(mdc_locks.c:835:mdc_finish_enqueue()) @@@ op=0x8 disposition=0xb, status=0 req@ffff9ada60625c00 x1875449616261120/t0(0) o101->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 576/688 e 0 to 0 dl 1788572473 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1788572457.905959:0:90306:0:(mdc_locks.c:1359:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572457.905974:0:90306:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff9ada43471800 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1788572457.905982:0:90306:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1788572457.906016:0:90306:0:(llite_lib.c:3119:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 69632 00000080:00010000:0.0:1788572457.906033:0:90306:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:0.0:1788572457.906039:0:90306:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572457.906043:0:90306:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec0d9bcb8 00010000:00010000:0.0:1788572457.906046:0:90306:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788572457.906056:0:90306:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200008:0.0:1788572457.906941:0:90306:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1788572457.906951:0:90306:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572457.906972:0:90306:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572457.906986:0:90306:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788572457.906998:0:90306:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1788572457.907006:0:90306:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572457.907012:0:90306:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572457.907016:0:90306:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:0.0:1788572457.907021:0:90306:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec0d9bcb8 00010000:00010000:0.0:1788572457.907024:0:90306:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1788572457.907032:0:90306:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788572457.907161:0:90306:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572457.907169:0:90306:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572457.907176:0:90306:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788572457.907185:0:90306:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:0.0:1788572457.907191:0:90306:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572457.907195:0:90306:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572457.907198:0:90306:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:0.0:1788572457.907201:0:90306:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec0d9bbb0 00010000:00010000:0.0:1788572457.907205:0:90306:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:0.0:1788572457.907212:0:90306:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:0.0:1788572457.907220:0:90306:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:0.0:1788572457.907228:0:90306:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200080:0.0:1788572457.907331:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:0.0:1788572457.907349:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572400 uarg=0000000000000000 00800000:00000004:0.0:1788572457.907357:0:90306:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: age 4326, max_age 4374 00000100:00100000:0.0:1788572457.907402:0:90306:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:90306:1875449616261248:192.168.202.113@tcp:41:lfs.0 00000100:00100000:0.0:1788572457.907504:0:90306:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada604feb40 going to sleep for 11 seconds 00000100:00100000:1.0:1788572457.912225:0:90306:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:90306:1875449616261248:192.168.202.113@tcp:41:lfs.0 00800000:00000004:1.0:1788572457.912262:0:90306:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: update ffff9ada4481d198 cache blocks 317325/353529 objects 1023687/1024000 00000080:00200080:1.0:1788572457.912373:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.912396:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00800000:00000004:1.0:1788572457.912405:0:90306:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: age 4326, max_age 4374 00000100:00100000:1.0:1788572457.912449:0:90306:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:90306:1875449616261376:192.168.202.113@tcp:41:lfs.0 00000100:00100000:1.0:1788572457.913467:0:90306:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60703540 going to sleep for 11 seconds 00000100:00100000:1.0:1788572457.921015:0:90306:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:90306:1875449616261376:192.168.202.113@tcp:41:lfs.0 00800000:00000004:1.0:1788572457.921122:0:90306:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: update ffff9ada44819e18 cache blocks 317392/353529 objects 1023736/1024000 00000080:00200080:1.0:1788572457.921407:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921430:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921440:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921445:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921449:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921453:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921459:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921463:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921466:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921470:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921474:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921478:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921482:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921486:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921490:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921494:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921498:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921502:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921506:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921509:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921513:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921517:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921522:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921525:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921529:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921532:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921551:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921556:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921560:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921564:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921569:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921573:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921577:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921581:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921585:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921589:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921593:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921596:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921600:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921604:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921608:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921612:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921615:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921619:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921624:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921627:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921631:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921635:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921639:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921654:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921660:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921664:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921668:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921671:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921675:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921679:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921683:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921686:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921691:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921694:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921698:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921702:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921706:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921709:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921714:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921717:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921721:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921725:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921729:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921732:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921736:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921739:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921744:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921748:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921752:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921755:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921759:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921762:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921767:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921771:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921774:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921779:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921782:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921786:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921789:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921793:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921797:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921801:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921804:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921808:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921812:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921815:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921819:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921833:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921844:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921849:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921889:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921894:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921898:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921901:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921905:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921908:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921911:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921925:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921930:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921934:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921938:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921941:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921945:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921948:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921952:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921954:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921957:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921961:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921964:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921967:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921971:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921974:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921978:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921981:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921985:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921988:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921992:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.921995:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.921999:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00800000:00000080:1.0:1788572457.922003:0:90306:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572457.922007:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:1.0:1788572457.922014:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00020000:00000004:1.0:1788572457.922032:0:90306:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: age 4326, max_age 4374 00000100:00080000:1.0:1788572457.922049:0:90306:0:(import.c:545:import_select_connection()) lustre-OST0000-osc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4326 00000100:00080000:1.0:1788572457.922064:0:90306:0:(import.c:665:import_select_connection()) lustre-OST0000-osc-ffff9ada58b0a000: import ffff9ada48407800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572457.922078:0:90306:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683394 ocd_grant: 8388608, lost: 0. 00000100:00100000:1.0:1788572457.922108:0:90306:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c000 x1875449616261504/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788572457.922203:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616261504:192.168.202.113@tcp:8:lfs.0 00000100:00080000:1.0:1788572457.922558:0:90306:0:(client.c:1846:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != CONNECTING) req@ffff9ada4566f800 x1875449616261632/t0(0) o13->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572457.922569:0:90306:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60703540 going to sleep for 0 seconds 00000100:00100000:3.0:1788572457.928095:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-ffff9ada58b0a000: committing for last_committed 8589935763 gen 7 00000100:00080000:3.0:1788572457.928117:0:2382:0:(import.c:1176:ptlrpc_connect_interpret()) lustre-OST0000-osc-ffff9ada58b0a000: connect to target with instance 3 00000100:00080000:3.0:1788572457.928127:0:2382:0:(import.c:1000:ptlrpc_connect_set_flags()) lustre-OST0000-osc-ffff9ada58b0a000: Resetting ns_connect_flags to server flags: 0xa0427af2e3440038 00000100:00080000:3.0:1788572457.928141:0:2382:0:(import.c:1257:ptlrpc_connect_interpret()) connected to replayable target: lustre-OST0000_UUID 00000100:00080000:3.0:1788572457.928145:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada48407800 lustre-OST0000_UUID: changing import state from CONNECTING to FULL 00000080:00000004:3.0:1788572457.928156:0:2382:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0xa0427af2e3440038 -> 0xa0427af2e3440038 00000080:00080000:3.0:1788572457.928160:0:2382:0:(lcommon_misc.c:56:cl_init_ea_size()) updating def/max_easize: 72/65536 00000100:00080000:3.0:1788572457.929052:0:2382:0:(recover.c:191:ptlrpc_wake_delayed()) @@@ waking (set 000000006b02a08e): req@ffff9ada4566f800 x1875449616261632/t0(0) o13->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572457.929086:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616261504:192.168.202.113@tcp:8:lfs.0 00000100:00100000:1.0:1788572457.931320:0:90306:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60703540 going to sleep for 11 seconds 00000100:00100000:1.0:1788572457.934253:0:90306:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:90306:1875449616261632:192.168.202.113@tcp:13:lfs.0 00020000:00000004:1.0:1788572457.934302:0:90306:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: update ffff9ada4354d198 cache blocks 901356/958279 objects 261728/262144 00000080:00200080:1.0:1788572457.934444:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:1.0:1788572457.934466:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00020000:00000004:1.0:1788572457.934476:0:90306:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: age 4326, max_age 4374 00000100:00080000:1.0:1788572457.934491:0:90306:0:(import.c:545:import_select_connection()) lustre-OST0001-osc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4326 00000100:00080000:1.0:1788572457.934505:0:90306:0:(import.c:665:import_select_connection()) lustre-OST0001-osc-ffff9ada58b0a000: import ffff9ada48400800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572457.934519:0:90306:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683394 ocd_grant: 8437760, lost: 0. 00000100:00100000:1.0:1788572457.934767:0:90306:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566dc00 x1875449616261760/t0(0) o8->lustre-OST0001-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572457.934815:0:90306:0:(client.c:1846:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != CONNECTING) req@ffff9ada4566f800 x1875449616261888/t0(0) o13->lustre-OST0001-osc-ffff9ada58b0a000@192.168.202.113@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572457.934826:0:90306:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60703540 going to sleep for 0 seconds 00000100:00100000:2.0:1788572457.935190:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616261760:192.168.202.113@tcp:8:lfs.0 00000100:00100000:2.0:1788572457.938433:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-ffff9ada58b0a000: committing for last_committed 4294967635 gen 8 00000100:00080000:2.0:1788572457.938456:0:2382:0:(import.c:1176:ptlrpc_connect_interpret()) lustre-OST0001-osc-ffff9ada58b0a000: connect to target with instance 1 00000100:00080000:2.0:1788572457.938469:0:2382:0:(import.c:1000:ptlrpc_connect_set_flags()) lustre-OST0001-osc-ffff9ada58b0a000: Resetting ns_connect_flags to server flags: 0xa0427af2e3440038 00000100:00080000:2.0:1788572457.938485:0:2382:0:(import.c:1257:ptlrpc_connect_interpret()) connected to replayable target: lustre-OST0001_UUID 00000100:00080000:2.0:1788572457.938490:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada48400800 lustre-OST0001_UUID: changing import state from CONNECTING to FULL 00000080:00000004:2.0:1788572457.938502:0:2382:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0xa0427af2e3440038 -> 0xa0427af2e3440038 00000080:00080000:2.0:1788572457.938508:0:2382:0:(lcommon_misc.c:56:cl_init_ea_size()) updating def/max_easize: 72/65536 00000100:00080000:2.0:1788572457.938526:0:2382:0:(recover.c:191:ptlrpc_wake_delayed()) @@@ waking (set 000000006b02a08e): req@ffff9ada4566f800 x1875449616261888/t0(0) o13->lustre-OST0001-osc-ffff9ada58b0a000@192.168.202.113@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788572457.938565:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616261760:192.168.202.113@tcp:8:lfs.0 00000100:00100000:0.0:1788572457.938837:0:90306:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60703540 going to sleep for 11 seconds 00000100:00100000:0.0:1788572457.941904:0:90306:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:90306:1875449616261888:192.168.202.113@tcp:13:lfs.0 00020000:00000004:0.0:1788572457.941951:0:90306:0:(obd_class.h:1116:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: update ffff9ada43549e18 cache blocks 901363/958279 objects 261750/262144 00000080:00200080:0.0:1788572457.942017:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942238:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942248:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942252:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942256:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942259:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942263:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942266:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942269:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942272:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942525:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942532:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942635:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942640:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942644:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942648:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942652:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942655:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942659:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942662:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942666:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942669:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942672:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942676:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942680:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942684:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942688:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942691:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942696:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942699:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942703:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942706:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942710:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942715:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942718:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942722:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942725:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942729:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942733:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942736:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942740:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942744:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942748:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942751:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942755:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942758:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942761:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942764:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942768:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942771:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942774:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942777:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942781:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942784:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942787:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942790:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942793:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942796:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942799:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942802:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942805:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942808:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942811:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942814:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942817:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942820:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942823:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942826:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942829:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942833:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942836:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942840:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942844:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942848:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942886:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942892:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942897:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942901:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942904:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942908:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942911:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942914:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942927:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942930:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942934:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942937:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942940:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942943:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942946:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942949:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942953:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942955:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942959:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942962:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942965:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942968:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942971:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942974:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942978:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942981:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942984:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942987:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942990:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942993:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.942996:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.942999:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.943002:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.943005:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.943008:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.943011:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.943014:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.943017:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.943020:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.943031:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.943034:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.943037:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.943041:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.943044:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.943047:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.943050:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.943053:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.943056:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.943060:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.943064:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572457.943067:0:90306:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffcdf94c700 00020000:00000080:0.0:1788572457.943070:0:90306:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200008:0.0:1788572457.943085:0:90306:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00010000:00010000:3.0:1788572458.871649:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572459.895127:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572460.922658:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572461.431328:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616262144:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572461.431455:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616262400:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572461.431493:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:3.0:1788572461.431499:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449616262016:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572461.431530:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616262272:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572461.431552:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616262528:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572461.440965:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449616262016:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572461.440995:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616262272:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572461.441007:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616262528:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572461.441020:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616262144:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572461.441031:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616262400:192.168.202.113@tcp:400:kworker.0 00010000:00010000:3.0:1788572461.943452:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572462.967242:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572463.991524:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572465.015289:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1788572465.706376:0:90588:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 00010000:00010000:2.0:1788572466.040599:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572466.553418:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449616262656:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572466.553525:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616262912:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572466.553557:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616263168:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572466.553593:0:2383:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1788572466.553600:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616262784:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572466.553644:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616263040:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572466.569982:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616262784:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572466.570007:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449616262656:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572466.570021:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616262912:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572466.578639:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616263040:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572466.578666:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616263168:192.168.202.113@tcp:400:kworker.0 00010000:00010000:1.0:1788572467.063081:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1788572467.867850:0:90736:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec 00010000:00010000:3.0:1788572468.089268:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572469.118087:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:1.0:1788572469.995176:0:90774:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1788572469.995200:0:90774:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:1.0:1788572469.995301:0:90774:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1788572469.995329:0:90774:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1788572469.995349:0:90774:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1788572469.995359:0:90774:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1788572469.995369:0:90774:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:1.0:1788572469.995373:0:90774:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:1.0:1788572469.995383:0:90774:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec0d17cb8 00010000:00010000:1.0:1788572469.995388:0:90774:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1788572469.995399:0:90774:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00200008:1.0:1788572469.996553:0:90774:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 0, flags 0, foreign 0 00000080:00200000:1.0:1788572469.996562:0:90774:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:1.0:1788572469.996582:0:90774:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1788572469.996597:0:90774:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:1.0:1788572469.996610:0:90774:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00000080:00200000:1.0:1788572469.996623:0:90774:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1788572469.996631:0:90774:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:1.0:1788572469.996634:0:90774:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:1.0:1788572469.996640:0:90774:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec0d17cb8 00010000:00010000:1.0:1788572469.996644:0:90774:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 1 00010000:00010000:1.0:1788572469.996653:0:90774:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:1.0:1788572469.996793:0:90774:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:1.0:1788572469.996802:0:90774:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1788572469.996810:0:90774:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:1.0:1788572469.996819:0:90774:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00000080:00200000:1.0:1788572469.996825:0:90774:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1788572469.996829:0:90774:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:1.0:1788572469.996832:0:90774:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:1.0:1788572469.996836:0:90774:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec0d17bb0 00010000:00010000:1.0:1788572469.996839:0:90774:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 1 00010000:00010000:1.0:1788572469.996846:0:90774:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000080:00200000:1.0:1788572469.997416:0:90774:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:1.0:1788572469.997429:0:90774:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200080:1.0:1788572469.997650:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572469.997673:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00800000:00000004:1.0:1788572469.997684:0:90774:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: age 4375, max_age 4386 00000100:00100000:1.0:1788572469.997742:0:90774:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:90774:1875449616263296:192.168.202.113@tcp:41:lfs.0 00000100:00100000:1.0:1788572469.997895:0:90774:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60a77b40 going to sleep for 11 seconds 00000100:00100000:1.0:1788572470.003205:0:90774:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:90774:1875449616263296:192.168.202.113@tcp:41:lfs.0 00800000:00000004:1.0:1788572470.003251:0:90774:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: update ffff9ada4481d198 cache blocks 317325/353529 objects 1023687/1024000 00000080:00200080:1.0:1788572470.003342:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.003362:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00800000:00000004:1.0:1788572470.003372:0:90774:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: age 4375, max_age 4386 00000100:00100000:1.0:1788572470.003419:0:90774:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:90774:1875449616263424:192.168.202.113@tcp:41:lfs.0 00000100:00100000:1.0:1788572470.003513:0:90774:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60a77b40 going to sleep for 11 seconds 00000100:00100000:1.0:1788572470.008405:0:90774:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:90774:1875449616263424:192.168.202.113@tcp:41:lfs.0 00800000:00000004:1.0:1788572470.008441:0:90774:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: update ffff9ada44819e18 cache blocks 317392/353529 objects 1023736/1024000 00000080:00200080:1.0:1788572470.008518:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008539:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008547:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008551:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008555:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008559:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008565:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008568:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008573:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008577:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008581:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008585:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008589:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008593:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008597:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008610:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008615:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008619:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008624:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008627:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008631:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008635:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008639:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008642:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008646:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008650:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008654:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008657:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008661:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008664:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008667:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008672:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008676:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008680:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008683:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008687:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008690:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008694:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008697:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008701:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008705:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008708:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008713:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008716:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008720:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008723:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008727:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008730:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008734:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008737:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008740:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008743:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008747:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008751:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008755:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008758:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008763:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008767:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008771:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008776:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008780:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008784:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008788:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008791:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008794:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008798:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008801:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008805:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008809:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008812:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008816:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008820:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008824:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008828:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008831:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008835:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008839:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008842:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008845:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008848:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008887:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008894:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008899:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008902:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008906:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008909:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008912:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008916:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008920:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008923:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008927:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008930:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008934:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008938:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008942:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008947:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008950:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008953:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008956:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008959:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008963:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008966:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008969:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008973:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.008977:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.008994:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.009001:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.009005:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.009010:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.009014:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.009018:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.009021:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.009026:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.009029:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.009033:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.009036:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.009040:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.009044:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.009048:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.009052:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.009056:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.009060:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.009064:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.009068:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.009072:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00800000:00000080:1.0:1788572470.009075:0:90774:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00000080:00200080:1.0:1788572470.009081:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:1.0:1788572470.009089:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00020000:00000004:1.0:1788572470.009099:0:90774:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: age 4375, max_age 4386 00000100:00100000:1.0:1788572470.009147:0:90774:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:90774:1875449616263552:192.168.202.113@tcp:13:lfs.0 00000100:00100000:1.0:1788572470.009244:0:90774:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60a77b40 going to sleep for 11 seconds 00000100:00100000:1.0:1788572470.013089:0:90774:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:90774:1875449616263552:192.168.202.113@tcp:13:lfs.0 00020000:00000004:1.0:1788572470.013133:0:90774:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: update ffff9ada4354d198 cache blocks 901356/958279 objects 261728/262144 00000080:00200080:1.0:1788572470.013221:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:1.0:1788572470.013252:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a400 uarg=0000000000000000 00020000:00000004:1.0:1788572470.013263:0:90774:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: age 4375, max_age 4386 00000100:00100000:1.0:1788572470.013316:0:90774:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:90774:1875449616263680:192.168.202.113@tcp:13:lfs.0 00000100:00100000:1.0:1788572470.013409:0:90774:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60a77b40 going to sleep for 11 seconds 00000100:00100000:2.0:1788572470.016945:0:90774:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:90774:1875449616263680:192.168.202.113@tcp:13:lfs.0 00020000:00000004:2.0:1788572470.016994:0:90774:0:(obd_class.h:1116:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: update ffff9ada43549e18 cache blocks 901363/958279 objects 261750/262144 00000080:00200080:2.0:1788572470.017080:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017103:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017112:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017116:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017120:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017124:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017128:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017131:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017135:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017139:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017142:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017146:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017150:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017153:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017157:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017160:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017163:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017169:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017172:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017176:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017180:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017184:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017187:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017190:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017194:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017198:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017202:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017205:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017208:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017212:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017216:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017219:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017222:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017226:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017230:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017233:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017247:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017252:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017256:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017259:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017263:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017266:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017271:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017275:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017279:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017282:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017286:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017289:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017292:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017295:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017298:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017301:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017304:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017308:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017311:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017314:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017318:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017322:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017325:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017329:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017333:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017336:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017340:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017344:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017348:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017351:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017355:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017359:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017362:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017365:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017369:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017372:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017376:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017380:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017386:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017389:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017393:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017396:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017399:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017402:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017406:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017409:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017413:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017417:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017420:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017423:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017427:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017430:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017434:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017438:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017441:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017445:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017449:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017453:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017456:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017460:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017463:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017466:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017469:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017473:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017476:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017479:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017483:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017486:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017490:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017494:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017498:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017502:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017506:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017509:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017513:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017516:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017520:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017523:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017526:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017530:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017533:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017537:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017540:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017544:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017547:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017551:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017555:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017558:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200080:2.0:1788572470.017562:0:90774:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe81d0d250 00020000:00000080:2.0:1788572470.017565:0:90774:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00000080:00200008:2.0:1788572470.017585:0:90774:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00010000:00010000:1.0:1788572470.135352:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572471.160116:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572471.675263:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449616263808:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572471.675402:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616264064:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572471.675434:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616264320:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572471.675467:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1788572471.675474:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616263936:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572471.675494:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616264192:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572471.678382:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616263936:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572471.678406:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449616263808:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572471.678418:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616264064:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572471.678430:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616264320:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572471.680209:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616264192:192.168.202.113@tcp:400:kworker.0 00010000:00010000:3.0:1788572472.184642:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572473.208372:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572474.231195:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572475.255227:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572476.279545:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788572476.793328:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616264576:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572476.793504:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449616264448:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572476.793621:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616264704:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572476.793658:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616264960:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572476.794190:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616264832:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572476.797211:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449616264448:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572476.797241:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616264704:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572476.797448:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616264960:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572476.812570:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616264576:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572476.816134:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616264832:192.168.202.113@tcp:400:kworker.0 00010000:00010000:3.0:1788572477.303104:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572478.331974:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572479.354072:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572480.375538:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572481.406429:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788572481.911444:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449616265088:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572481.911574:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616265344:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572481.911604:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616265600:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572481.911637:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1788572481.911644:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616265216:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572481.911666:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616265472:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572481.923491:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616265216:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572481.923519:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616265472:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572481.923531:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449616265088:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572481.923543:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616265344:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572481.923555:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616265600:192.168.202.113@tcp:400:kworker.0 00010000:00010000:0.0:1788572482.425727:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572483.453436:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572484.473448:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572485.495024:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572486.520397:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572487.042214:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449616265728:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572487.042538:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616265984:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572487.042572:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616266240:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572487.042606:0:2383:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1788572487.042611:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616265856:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572487.042632:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616266112:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572487.053122:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616265856:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572487.053148:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616265984:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572487.056799:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616266112:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572487.056823:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616266240:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572487.063092:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449616265728:192.168.202.113@tcp:400:kworker.0 00010000:00010000:0.0:1788572487.557192:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572488.567507:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572489.594538:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1788572489.860689:0:91367:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == replay-single test 44a: race in target handle connect ========================================================== 21:41:27 (1788572487) 00010000:00010000:0.0:1788572490.627789:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572491.641938:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788572492.151160:0:58077:0:(import.c:65:import_set_state_nolock()) ffff9ada48407800 lustre-OST0000_UUID: changing import state from FULL to CONNECTING 00000100:00080000:0.0:1788572492.151168:0:58077:0:(import.c:2147:ptlrpc_disconnect_and_idle_import()) lustre-OST0000-osc-ffff9ada58b0a000: disconnect after 22s idle 00000100:00080000:0.0:1788572492.151192:0:58077:0:(import.c:65:import_set_state_nolock()) ffff9ada48400800 lustre-OST0001_UUID: changing import state from FULL to CONNECTING 00000100:00080000:0.0:1788572492.151195:0:58077:0:(import.c:2147:ptlrpc_disconnect_and_idle_import()) lustre-OST0001-osc-ffff9ada58b0a000: disconnect after 22s idle 00000100:00100000:2.0:1788572492.151331:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616266752:192.168.202.113@tcp:9:kworker.0 00000100:00100000:1.0:1788572492.151378:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449616266368:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572492.151476:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616266624:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572492.151476:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616266880:192.168.202.113@tcp:9:kworker.0 00000100:00100000:1.0:1788572492.151510:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1788572492.151517:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616266496:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572492.160670:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616266496:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572492.160705:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616266624:192.168.202.113@tcp:400:kworker.0 00000100:00080000:3.0:1788572492.161217:0:2382:0:(import.c:2057:ptlrpc_disconnect_idle_interpret()) @@@ inflight=1, refcount=3: rc = 0 req@ffff9ada4557f480 x1875449616266752/t0(0) o9->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572502 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572492.161230:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada48407800 lustre-OST0000_UUID: changing import state from CONNECTING to IDLE 00000100:00100000:3.0:1788572492.161234:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616266752:192.168.202.113@tcp:9:kworker.0 00000100:00080000:3.0:1788572492.161247:0:2382:0:(import.c:2057:ptlrpc_disconnect_idle_interpret()) @@@ inflight=1, refcount=3: rc = 0 req@ffff9ada4557c000 x1875449616266880/t0(0) o9->lustre-OST0001-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572502 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572492.161255:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada48400800 lustre-OST0001_UUID: changing import state from CONNECTING to IDLE 00000100:00100000:3.0:1788572492.161258:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616266880:192.168.202.113@tcp:9:kworker.0 00000100:00100000:1.0:1788572492.170034:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449616266368:192.168.202.113@tcp:400:kworker.0 00010000:00010000:3.0:1788572492.669939:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572493.689480:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572494.711464:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572495.737474:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:0.0:1788572496.434648:0:91505:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=40086685 len=0 uarg=00007ffdb4d67270 00000002:00000080:0.0:1788572496.434675:0:91505:0:(mdc_request.c:2317:mdc_iocontrol()) lustre-MDT0000-mdc-ffff9ada58b0a000: cmd=40086685 len=576 karg=ffff9ada45570c00 uarg=0000000000000000 00000100:00080000:0.0:1788572496.434685:0:91505:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:0.0:1788572496.434696:0:91505:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff9ada58b0a000: Connection to lustre-MDT0000 (at 192.168.202.113@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:0.0:1788572496.434705:0:91505:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572496.434712:0:91505:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4368 00000100:00080000:0.0:1788572496.434731:0:91505:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788572496.434746:0:91505:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788572496.434784:0:91505:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616267008/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788572496.434816:0:91505:0:(recover.c:337:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery started, waiting 20 seconds 00000100:00100000:3.0:1788572496.438026:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616267008:192.168.202.113@tcp:38:lctl.0 00010000:00010000:1.0:1788572496.760223:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788572497.276841:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572497.276850:0:58077:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1788572497.277439:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449616267136:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572497.277576:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1788572497.277584:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616267264:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572497.281840:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616267264:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572497.282303:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449616267136:192.168.202.113@tcp:400:kworker.0 00010000:00010000:3.0:1788572497.783695:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572498.807801:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572499.834630:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572500.855769:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788572501.493271:0:2382:0:(import.c:1176:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to target with instance 52 00000100:00080000:2.0:1788572501.493315:0:2382:0:(import.c:1341:ptlrpc_connect_interpret()) reconnected to lustre-MDT0000_UUID@192.168.202.113@tcp after partition 00000100:00080000:2.0:1788572501.493320:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to RECOVER 00000100:00080000:2.0:1788572501.493328:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from RECOVER to FULL 00000100:02000000:3.0:1788572501.495017:0:2382:0:(import.c:1787:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff9ada58b0a000: Connection restored to 192.168.202.113@tcp (at 192.168.202.113@tcp) 00000100:00100000:3.0:1788572501.495052:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616267008:192.168.202.113@tcp:38:lctl.0 00000100:00080000:0.0:1788572501.495626:0:91505:0:(recover.c:348:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery finished FULL, rc = 0 00000080:00200008:1.0:1788572501.523372:0:91508:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1788572501.523394:0:91508:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:1.0:1788572501.523427:0:91508:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1788572501.523453:0:91508:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:1.0:1788572501.523472:0:91508:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00000080:00200000:1.0:1788572501.523482:0:91508:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1788572501.523492:0:91508:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:1.0:1788572501.523495:0:91508:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:1.0:1788572501.523504:0:91508:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1c5fcb8 00010000:00010000:1.0:1788572501.523510:0:91508:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 1 00010000:00010000:1.0:1788572501.523520:0:91508:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00000080:00200008:1.0:1788572501.529424:0:91508:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 0, flags 0, foreign 0 00000080:00200000:1.0:1788572501.529441:0:91508:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:1.0:1788572501.529478:0:91508:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1788572501.529504:0:91508:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00010000:00010000:1.0:1788572501.529522:0:91508:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00000080:00200000:1.0:1788572501.529532:0:91508:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1788572501.529542:0:91508:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:1.0:1788572501.529547:0:91508:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:1.0:1788572501.529557:0:91508:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1c5fcb8 00010000:00010000:1.0:1788572501.529561:0:91508:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 7 lru_type: 1 00010000:00010000:1.0:1788572501.529572:0:91508:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00000080:00200000:1.0:1788572501.531167:0:91508:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:1.0:1788572501.531204:0:91508:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1788572501.531226:0:91508:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:1.0:1788572501.531244:0:91508:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00000080:00200000:1.0:1788572501.531253:0:91508:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1788572501.531261:0:91508:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:1.0:1788572501.531265:0:91508:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:1.0:1788572501.531272:0:91508:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1c5fbb0 00010000:00010000:1.0:1788572501.531277:0:91508:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 8 lru_type: 1 00010000:00010000:1.0:1788572501.531285:0:91508:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00000080:00200000:1.0:1788572501.531301:0:91508:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:1.0:1788572501.531312:0:91508:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200080:1.0:1788572501.531439:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:1.0:1788572501.531458:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648000 uarg=0000000000000000 00800000:00000004:1.0:1788572501.531466:0:91508:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: age 4387, max_age 4418 00000100:00100000:1.0:1788572501.531520:0:91508:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91508:1875449616267392:192.168.202.113@tcp:41:lfs.0 00000100:00100000:1.0:1788572501.531645:0:91508:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada50a4f540 going to sleep for 12 seconds 00000100:00100000:1.0:1788572501.542943:0:91508:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91508:1875449616267392:192.168.202.113@tcp:41:lfs.0 00800000:00000004:1.0:1788572501.542985:0:91508:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: update ffff9ada4481d198 cache blocks 317325/353529 objects 1023687/1024000 00000080:00200080:2.0:1788572501.543309:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.543336:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00800000:00000004:2.0:1788572501.543347:0:91508:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: age 4387, max_age 4418 00000100:00100000:2.0:1788572501.543396:0:91508:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91508:1875449616267520:192.168.202.113@tcp:41:lfs.0 00000100:00100000:2.0:1788572501.543640:0:91508:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6076ccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572501.547062:0:91508:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91508:1875449616267520:192.168.202.113@tcp:41:lfs.0 00800000:00000004:2.0:1788572501.547100:0:91508:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: update ffff9ada44819e18 cache blocks 317392/353529 objects 1023736/1024000 00000080:00200080:2.0:1788572501.547191:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547212:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547221:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547226:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547230:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547234:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547238:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547242:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547246:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547250:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547255:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547259:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547263:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547266:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547270:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547283:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547289:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547293:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547297:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547302:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547306:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547310:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547314:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547318:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547322:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547326:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547330:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547333:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547337:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547341:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547345:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547348:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547352:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547356:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547360:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547364:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547368:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547373:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547377:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547381:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547385:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547389:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547393:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547398:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547403:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547407:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547412:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547416:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547420:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547424:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547429:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547432:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547436:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547440:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547444:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547448:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547452:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547456:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547461:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547465:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547469:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547473:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547478:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547482:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547501:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547506:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547510:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547513:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547517:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547521:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547525:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547529:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547533:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547536:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547541:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547544:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547549:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547553:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547557:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547562:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547566:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547570:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547575:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547579:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547583:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547587:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547591:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547596:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547599:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547603:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547606:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547610:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547613:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547617:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547622:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547625:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547796:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547803:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547808:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547813:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547817:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547821:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547825:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547830:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547835:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547838:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.547843:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.547847:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.548076:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.548086:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.548094:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.548100:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.548104:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.548108:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.548113:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.548116:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.548119:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.548123:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.548128:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.548132:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.548135:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.548139:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.548143:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.548146:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.548149:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00800000:00000080:2.0:1788572501.548153:0:91508:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.548157:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.548164:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00020000:00000004:2.0:1788572501.548173:0:91508:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: age 4387, max_age 4418 00000100:00080000:2.0:1788572501.548191:0:91508:0:(import.c:545:import_select_connection()) lustre-OST0000-osc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4375 00000100:00080000:2.0:1788572501.548206:0:91508:0:(import.c:665:import_select_connection()) lustre-OST0000-osc-ffff9ada58b0a000: import ffff9ada48407800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572501.548219:0:91508:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683394 ocd_grant: 8388608, lost: 0. 00000100:00100000:2.0:1788572501.548250:0:91508:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624700 x1875449616267648/t0(0) o8->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788572501.548308:0:91508:0:(client.c:1846:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != CONNECTING) req@ffff9ada60626680 x1875449616267776/t0(0) o13->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788572501.548318:0:91508:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6076ccc0 going to sleep for 0 seconds 00000100:00100000:0.0:1788572501.548358:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616267648:192.168.202.113@tcp:8:lfs.0 00000100:00100000:0.0:1788572501.563448:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-ffff9ada58b0a000: committing for last_committed 8589935763 gen 8 00000100:00080000:0.0:1788572501.563473:0:2382:0:(import.c:1176:ptlrpc_connect_interpret()) lustre-OST0000-osc-ffff9ada58b0a000: connect to target with instance 3 00000100:00080000:0.0:1788572501.563484:0:2382:0:(import.c:1000:ptlrpc_connect_set_flags()) lustre-OST0000-osc-ffff9ada58b0a000: Resetting ns_connect_flags to server flags: 0xa0427af2e3440038 00000100:00080000:0.0:1788572501.563500:0:2382:0:(import.c:1257:ptlrpc_connect_interpret()) connected to replayable target: lustre-OST0000_UUID 00000100:00080000:0.0:1788572501.563504:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada48407800 lustre-OST0000_UUID: changing import state from CONNECTING to FULL 00000080:00000004:0.0:1788572501.563515:0:2382:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0xa0427af2e3440038 -> 0xa0427af2e3440038 00000080:00080000:0.0:1788572501.563521:0:2382:0:(lcommon_misc.c:56:cl_init_ea_size()) updating def/max_easize: 72/65536 00000100:00080000:0.0:1788572501.563539:0:2382:0:(recover.c:191:ptlrpc_wake_delayed()) @@@ waking (set 00000000ad8cb118): req@ffff9ada60626680 x1875449616267776/t0(0) o13->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572501.563573:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616267648:192.168.202.113@tcp:8:lfs.0 00000100:00100000:2.0:1788572501.563792:0:91508:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6076ccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572501.568441:0:91508:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91508:1875449616267776:192.168.202.113@tcp:13:lfs.0 00020000:00000004:2.0:1788572501.568483:0:91508:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: update ffff9ada4354d198 cache blocks 901356/958279 objects 261728/262144 00000080:00200080:2.0:1788572501.568593:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.568618:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00020000:00000004:2.0:1788572501.568630:0:91508:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: age 4387, max_age 4418 00000100:00080000:2.0:1788572501.568649:0:91508:0:(import.c:545:import_select_connection()) lustre-OST0001-osc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4375 00000100:00080000:2.0:1788572501.568666:0:91508:0:(import.c:665:import_select_connection()) lustre-OST0001-osc-ffff9ada58b0a000: import ffff9ada48400800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572501.568679:0:91508:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 34683394 ocd_grant: 8437760, lost: 0. 00000100:00100000:2.0:1788572501.568709:0:91508:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627100 x1875449616267904/t0(0) o8->lustre-OST0001-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788572501.568760:0:91508:0:(client.c:1846:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != CONNECTING) req@ffff9ada60626680 x1875449616268032/t0(0) o13->lustre-OST0001-osc-ffff9ada58b0a000@192.168.202.113@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788572501.568770:0:91508:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6076ccc0 going to sleep for 0 seconds 00000100:00100000:0.0:1788572501.568944:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616267904:192.168.202.113@tcp:8:lfs.0 00000100:00100000:0.0:1788572501.580090:0:2382:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-ffff9ada58b0a000: committing for last_committed 4294967635 gen 9 00000100:00080000:0.0:1788572501.580115:0:2382:0:(import.c:1176:ptlrpc_connect_interpret()) lustre-OST0001-osc-ffff9ada58b0a000: connect to target with instance 1 00000100:00080000:0.0:1788572501.580127:0:2382:0:(import.c:1000:ptlrpc_connect_set_flags()) lustre-OST0001-osc-ffff9ada58b0a000: Resetting ns_connect_flags to server flags: 0xa0427af2e3440038 00000100:00080000:0.0:1788572501.580144:0:2382:0:(import.c:1257:ptlrpc_connect_interpret()) connected to replayable target: lustre-OST0001_UUID 00000100:00080000:0.0:1788572501.580150:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada48400800 lustre-OST0001_UUID: changing import state from CONNECTING to FULL 00000080:00000004:0.0:1788572501.580164:0:2382:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0xa0427af2e3440038 -> 0xa0427af2e3440038 00000080:00080000:0.0:1788572501.580170:0:2382:0:(lcommon_misc.c:56:cl_init_ea_size()) updating def/max_easize: 72/65536 00000100:00080000:0.0:1788572501.580190:0:2382:0:(recover.c:191:ptlrpc_wake_delayed()) @@@ waking (set 00000000ad8cb118): req@ffff9ada60626680 x1875449616268032/t0(0) o13->lustre-OST0001-osc-ffff9ada58b0a000@192.168.202.113@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572501.580226:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616267904:192.168.202.113@tcp:8:lfs.0 00000100:00100000:2.0:1788572501.581509:0:91508:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6076ccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788572501.590080:0:91508:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91508:1875449616268032:192.168.202.113@tcp:13:lfs.0 00020000:00000004:2.0:1788572501.590124:0:91508:0:(obd_class.h:1116:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: update ffff9ada43549e18 cache blocks 901363/958279 objects 261734/262144 00000080:00200080:2.0:1788572501.590220:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590244:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590264:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590271:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590276:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590281:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590285:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590288:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590292:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590294:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590298:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590301:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590305:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590309:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590313:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590317:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590319:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590322:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590326:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590330:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590334:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590337:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590341:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590344:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590348:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590351:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590355:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590358:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590361:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590364:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590368:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590371:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590374:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590377:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590380:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590383:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590387:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590390:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590393:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590396:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590400:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590404:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590407:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590410:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590414:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590417:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590420:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590423:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590426:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590429:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590432:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590436:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590439:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590443:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590447:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590450:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590453:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590457:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590460:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590464:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590468:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590472:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590476:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590480:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590483:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590486:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590489:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590492:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590495:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590499:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590502:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590506:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590509:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590513:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590517:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590520:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590523:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590526:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590530:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590533:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590537:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590540:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590543:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590546:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590550:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590553:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590557:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590560:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590563:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590567:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590571:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590574:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590578:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590582:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590585:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590589:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590593:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590596:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590601:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590604:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590608:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590611:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590615:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590618:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590623:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590626:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590631:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590635:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590638:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.590642:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.590645:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.591865:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.591893:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.591899:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.591905:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.591909:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.591913:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.591917:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.591921:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.591924:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.591928:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.591932:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.591935:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.591939:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200080:2.0:1788572501.591943:0:91508:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fffe92dd220 00020000:00000080:2.0:1788572501.591946:0:91508:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada4566ac00 uarg=0000000000000000 00000080:00200008:2.0:1788572501.591965:0:91508:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00010000:00010000:3.0:1788572501.880609:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572502.391970:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616268288:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572502.392110:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616268544:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572502.392150:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:3.0:1788572502.392157:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449616268160:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572502.392190:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616268416:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572502.392213:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616268672:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572502.394666:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616268288:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572502.395293:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449616268160:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572502.408913:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616268672:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572502.408936:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616268544:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572502.423399:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616268416:192.168.202.113@tcp:400:kworker.0 00010000:00010000:1.0:1788572502.904299:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:2.0:1788572503.728844:0:91526:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=40086685 len=0 uarg=00007fffa684cda0 00000002:00000080:2.0:1788572503.728915:0:91526:0:(mdc_request.c:2317:mdc_iocontrol()) lustre-MDT0000-mdc-ffff9ada58b0a000: cmd=40086685 len=576 karg=ffff9ada45669800 uarg=0000000000000000 00000100:00080000:2.0:1788572503.728926:0:91526:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:2.0:1788572503.728937:0:91526:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff9ada58b0a000: Connection to lustre-MDT0000 (at 192.168.202.113@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:2.0:1788572503.728946:0:91526:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572503.728951:0:91526:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4414 00000100:00080000:2.0:1788572503.728971:0:91526:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572503.728987:0:91526:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788572503.729036:0:91526:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616268800/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788572503.729075:0:91526:0:(recover.c:337:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery started, waiting 20 seconds 00000100:00100000:3.0:1788572503.729250:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616268800:192.168.202.113@tcp:38:lctl.0 00010000:00010000:0.0:1788572503.927211:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572504.953511:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572505.975315:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572507.000729:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788572507.514345:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572507.514353:0:58077:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1788572507.514440:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449616268928:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572507.514450:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616269056:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572507.514559:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616269184:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572507.514641:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616269312:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572507.523595:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616269312:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572507.524016:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449616268928:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572507.540891:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616269184:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572507.545220:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616269056:192.168.202.113@tcp:400:kworker.0 00010000:00010000:3.0:1788572508.023431:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572509.047292:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788572509.180702:0:2382:0:(import.c:1176:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to target with instance 52 00000100:00080000:3.0:1788572509.180756:0:2382:0:(import.c:1341:ptlrpc_connect_interpret()) reconnected to lustre-MDT0000_UUID@192.168.202.113@tcp after partition 00000100:00080000:3.0:1788572509.180761:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to RECOVER 00000100:00080000:3.0:1788572509.180770:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from RECOVER to FULL 00000100:02000000:1.0:1788572509.181423:0:2382:0:(import.c:1787:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff9ada58b0a000: Connection restored to 192.168.202.113@tcp (at 192.168.202.113@tcp) 00000100:00100000:1.0:1788572509.181441:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616268800:192.168.202.113@tcp:38:lctl.0 00000100:00080000:2.0:1788572509.181645:0:91526:0:(recover.c:348:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery finished FULL, rc = 0 00000080:00200008:3.0:1788572509.232529:0:91529:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1788572509.232548:0:91529:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:3.0:1788572509.232581:0:91529:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1788572509.232634:0:91529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00010000:00010000:3.0:1788572509.232660:0:91529:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00000080:00200000:3.0:1788572509.232668:0:91529:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1788572509.232678:0:91529:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:3.0:1788572509.232682:0:91529:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:3.0:1788572509.232692:0:91529:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1dc7cb8 00010000:00010000:3.0:1788572509.232696:0:91529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 9 lru_type: 1 00010000:00010000:3.0:1788572509.232707:0:91529:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00000080:00200008:3.0:1788572509.233483:0:91529:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 0, flags 0, foreign 0 00000080:00200000:3.0:1788572509.233490:0:91529:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:3.0:1788572509.233504:0:91529:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1788572509.233532:0:91529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00010000:00010000:3.0:1788572509.233545:0:91529:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00000080:00200000:3.0:1788572509.233552:0:91529:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1788572509.233558:0:91529:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:3.0:1788572509.233562:0:91529:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:3.0:1788572509.233567:0:91529:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1dc7cb8 00010000:00010000:3.0:1788572509.233571:0:91529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 10 lru_type: 1 00010000:00010000:3.0:1788572509.233579:0:91529:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00000080:00200000:3.0:1788572509.233712:0:91529:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:3.0:1788572509.233721:0:91529:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1788572509.233730:0:91529:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00010000:00010000:3.0:1788572509.233744:0:91529:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00000080:00200000:3.0:1788572509.233751:0:91529:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1788572509.233755:0:91529:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:3.0:1788572509.233757:0:91529:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:3.0:1788572509.233761:0:91529:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1dc7bb0 00010000:00010000:3.0:1788572509.233764:0:91529:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 11 lru_type: 1 00010000:00010000:3.0:1788572509.233771:0:91529:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 12 lru_type: 1 00000080:00200000:3.0:1788572509.233779:0:91529:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:3.0:1788572509.233788:0:91529:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200080:3.0:1788572509.233951:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:3.0:1788572509.233972:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45536000 uarg=0000000000000000 00800000:00000004:3.0:1788572509.233980:0:91529:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: age 4419, max_age 4425 00000100:00100000:3.0:1788572509.234060:0:91529:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91529:1875449616269440:192.168.202.113@tcp:41:lfs.0 00000100:00100000:3.0:1788572509.234186:0:91529:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2780 going to sleep for 12 seconds 00000100:00100000:1.0:1788572509.250240:0:91529:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91529:1875449616269440:192.168.202.113@tcp:41:lfs.0 00800000:00000004:1.0:1788572509.250278:0:91529:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: update ffff9ada4481d198 cache blocks 317325/353529 objects 1023687/1024000 00000080:00200080:1.0:1788572509.250383:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.250405:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00800000:00000004:1.0:1788572509.250415:0:91529:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: age 4419, max_age 4425 00000100:00100000:1.0:1788572509.250459:0:91529:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91529:1875449616269568:192.168.202.113@tcp:41:lfs.0 00000100:00100000:1.0:1788572509.250542:0:91529:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada50a4f540 going to sleep for 12 seconds 00000100:00100000:1.0:1788572509.254463:0:91529:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91529:1875449616269568:192.168.202.113@tcp:41:lfs.0 00800000:00000004:1.0:1788572509.254500:0:91529:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: update ffff9ada44819e18 cache blocks 317392/353529 objects 1023736/1024000 00000080:00200080:1.0:1788572509.254585:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.254606:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.254615:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.254619:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.254623:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.254627:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.254631:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.254635:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.254639:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.254642:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.254646:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.254650:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.254655:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.254658:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.254663:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.254666:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.254670:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.254674:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.254678:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.254682:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.254686:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.254690:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.254694:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.254698:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.254702:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.254706:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.254710:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.254714:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.254718:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.254721:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.254726:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.254730:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.254734:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.254738:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.254742:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.254745:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.254749:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.254753:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.254757:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.254761:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.254765:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.254769:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.254773:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.254776:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.254780:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.254783:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.254787:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.254790:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.254794:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.254797:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.254801:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.254804:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.254808:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.254812:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.254815:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.254819:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.254823:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.254826:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.254830:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.254834:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.254837:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.254841:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.254846:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.254849:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.254905:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.254911:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.254916:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.254919:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.254924:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.254927:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.254930:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.254933:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.254990:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.254996:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.255014:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.255018:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.255022:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.255026:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.255029:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.255032:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.255035:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.255038:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.255042:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.255045:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.255049:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.255053:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.255057:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.255061:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.255064:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.255067:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.255071:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.255074:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.255077:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.255081:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.255084:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.255088:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.255091:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.255094:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.255098:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.255101:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.255104:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.255107:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.255110:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.255113:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.255116:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.255119:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.255123:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.255126:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.255129:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.255132:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.255135:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.255138:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.255142:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.255145:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.255148:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.255152:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.255156:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.255159:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.255163:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.255166:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.255170:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.255173:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.255177:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.255180:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.255183:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00800000:00000080:1.0:1788572509.255186:0:91529:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.255191:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.255201:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00020000:00000004:1.0:1788572509.255210:0:91529:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: age 4419, max_age 4425 00000100:00100000:1.0:1788572509.255261:0:91529:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91529:1875449616269696:192.168.202.113@tcp:13:lfs.0 00000100:00100000:1.0:1788572509.255362:0:91529:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada50a4f540 going to sleep for 11 seconds 00000100:00100000:1.0:1788572509.267316:0:91529:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91529:1875449616269696:192.168.202.113@tcp:13:lfs.0 00020000:00000004:1.0:1788572509.267359:0:91529:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: update ffff9ada4354d198 cache blocks 901356/958279 objects 261728/262144 00000080:00200080:1.0:1788572509.267566:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.267590:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00020000:00000004:1.0:1788572509.267599:0:91529:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: age 4419, max_age 4425 00000100:00100000:1.0:1788572509.267648:0:91529:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91529:1875449616269824:192.168.202.113@tcp:13:lfs.0 00000100:00100000:1.0:1788572509.267763:0:91529:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada50a4f540 going to sleep for 11 seconds 00000100:00100000:1.0:1788572509.283795:0:91529:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91529:1875449616269824:192.168.202.113@tcp:13:lfs.0 00020000:00000004:1.0:1788572509.283836:0:91529:0:(obd_class.h:1116:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: update ffff9ada43549e18 cache blocks 901363/958279 objects 261734/262144 00000080:00200080:1.0:1788572509.284135:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284171:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284182:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284187:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284192:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284196:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284200:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284203:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284208:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284211:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284215:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284218:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284222:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284225:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284228:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284232:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284235:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284239:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284242:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284257:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284265:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284269:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284273:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284277:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284281:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284284:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284288:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284292:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284296:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284300:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284304:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284308:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284312:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284316:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284321:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284324:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284329:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284333:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284337:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284341:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284345:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284349:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284353:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284357:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284361:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284365:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284369:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284373:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284376:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284380:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284383:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284387:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284391:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284395:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284527:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284534:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284538:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284542:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284546:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284549:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284553:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284557:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284561:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284564:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284568:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284572:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284576:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284579:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284583:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284586:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284591:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284594:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284598:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284601:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284606:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284609:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284613:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284617:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284621:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284624:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284628:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284632:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284636:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284640:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284643:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284646:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284650:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284653:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284657:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284661:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284664:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284668:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284672:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284676:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284680:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284684:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284687:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284690:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284694:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284697:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284700:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284704:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284707:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284711:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284715:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284718:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284847:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284890:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284899:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284903:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284907:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284911:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284915:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284918:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284922:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284926:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284929:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.284932:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.284936:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.285188:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.285197:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.285201:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.285205:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.285209:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200080:1.0:1788572509.285212:0:91529:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7fff6f8ea120 00020000:00000080:1.0:1788572509.285216:0:91529:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164a800 uarg=0000000000000000 00000080:00200008:1.0:1788572509.285234:0:91529:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00010000:00010000:0.0:1788572510.072745:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572511.096242:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:1.0:1788572511.890154:0:91547:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=40086685 len=0 uarg=00007ffc1503ebe0 00000002:00000080:1.0:1788572511.890178:0:91547:0:(mdc_request.c:2317:mdc_iocontrol()) lustre-MDT0000-mdc-ffff9ada58b0a000: cmd=40086685 len=576 karg=ffff9ada51648000 uarg=0000000000000000 00000100:00080000:1.0:1788572511.890187:0:91547:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:1.0:1788572511.890197:0:91547:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff9ada58b0a000: Connection to lustre-MDT0000 (at 192.168.202.113@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:1.0:1788572511.890205:0:91547:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572511.890210:0:91547:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4421 00000100:00080000:1.0:1788572511.890229:0:91547:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572511.890242:0:91547:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572511.890274:0:91547:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d180 x1875449616269952/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572511.890302:0:91547:0:(recover.c:337:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery started, waiting 20 seconds 00000100:00100000:3.0:1788572511.890502:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616269952:192.168.202.113@tcp:38:lctl.0 00010000:00010000:1.0:1788572512.119723:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788572512.636440:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572512.636448:0:85951:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1788572512.636542:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449616270080:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572512.636660:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616270336:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572512.636707:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1788572512.636714:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616270208:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572512.636738:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616270464:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572512.652978:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616270208:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572512.653012:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616270464:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572512.653025:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449616270080:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572512.653038:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616270336:192.168.202.113@tcp:400:kworker.0 00010000:00010000:0.0:1788572513.143404:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788572513.655082:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572513.655091:0:85951:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1788572513.658251:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616270720:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572513.658435:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616270976:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572513.658480:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1788572513.658579:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449616270592:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572513.659573:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616270848:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572513.685172:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616270976:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572513.685336:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449616270592:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572513.685352:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616270848:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572513.685368:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616270720:192.168.202.113@tcp:400:kworker.0 00010000:00010000:3.0:1788572514.167438:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572515.192097:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572516.230375:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572517.241496:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788572517.372552:0:2382:0:(import.c:1176:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to target with instance 52 00000100:00080000:1.0:1788572517.372584:0:2382:0:(import.c:1341:ptlrpc_connect_interpret()) reconnected to lustre-MDT0000_UUID@192.168.202.113@tcp after partition 00000100:00080000:1.0:1788572517.372702:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to RECOVER 00000100:00080000:1.0:1788572517.372709:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from RECOVER to FULL 00000100:02000000:1.0:1788572517.373831:0:2382:0:(import.c:1787:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff9ada58b0a000: Connection restored to 192.168.202.113@tcp (at 192.168.202.113@tcp) 00000100:00080000:2.0:1788572517.379571:0:91547:0:(recover.c:348:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery finished FULL, rc = 0 00000100:00100000:1.0:1788572517.382604:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616269952:192.168.202.113@tcp:38:lctl.0 00000080:00200008:0.0:1788572517.418266:0:91550:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1788572517.418281:0:91550:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572517.418310:0:91550:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572517.418345:0:91550:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 12 lru_type: 1 00010000:00010000:0.0:1788572517.418363:0:91550:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 12 lru_type: 1 00000080:00200000:0.0:1788572517.418372:0:91550:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572517.418381:0:91550:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572517.418385:0:91550:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:0.0:1788572517.418395:0:91550:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1e4fcb8 00010000:00010000:0.0:1788572517.418399:0:91550:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 12 lru_type: 1 00010000:00010000:0.0:1788572517.418409:0:91550:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 13 lru_type: 1 00000080:00200008:0.0:1788572517.419062:0:91550:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1788572517.419068:0:91550:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572517.419081:0:91550:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572517.419090:0:91550:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 13 lru_type: 1 00010000:00010000:0.0:1788572517.419099:0:91550:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 13 lru_type: 1 00000080:00200000:0.0:1788572517.419105:0:91550:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572517.419110:0:91550:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572517.419113:0:91550:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:0.0:1788572517.419117:0:91550:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1e4fcb8 00010000:00010000:0.0:1788572517.419120:0:91550:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 13 lru_type: 1 00010000:00010000:0.0:1788572517.419127:0:91550:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 14 lru_type: 1 00000080:00200000:0.0:1788572517.419227:0:91550:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572517.419236:0:91550:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572517.419258:0:91550:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 14 lru_type: 1 00010000:00010000:0.0:1788572517.419267:0:91550:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 14 lru_type: 1 00000080:00200000:0.0:1788572517.419278:0:91550:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572517.419282:0:91550:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572517.419285:0:91550:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:0.0:1788572517.419289:0:91550:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1e4fbb0 00010000:00010000:0.0:1788572517.419292:0:91550:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 14 lru_type: 1 00010000:00010000:0.0:1788572517.419299:0:91550:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 15 lru_type: 1 00000080:00200000:0.0:1788572517.419306:0:91550:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:0.0:1788572517.419313:0:91550:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200080:0.0:1788572517.419409:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.419425:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00800000:00000004:0.0:1788572517.419433:0:91550:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: age 4426, max_age 4434 00000100:00100000:0.0:1788572517.419482:0:91550:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91550:1875449616271104:192.168.202.113@tcp:41:lfs.0 00000100:00100000:0.0:1788572517.419568:0:91550:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada604fe3c0 going to sleep for 12 seconds 00000100:00100000:0.0:1788572517.430703:0:91550:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91550:1875449616271104:192.168.202.113@tcp:41:lfs.0 00800000:00000004:0.0:1788572517.430736:0:91550:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: update ffff9ada4481d198 cache blocks 317325/353529 objects 1023687/1024000 00000080:00200080:0.0:1788572517.430829:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.430850:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00800000:00000004:0.0:1788572517.430882:0:91550:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: age 4426, max_age 4434 00000100:00100000:0.0:1788572517.430920:0:91550:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91550:1875449616271232:192.168.202.113@tcp:41:lfs.0 00000100:00100000:0.0:1788572517.431021:0:91550:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada604fe3c0 going to sleep for 12 seconds 00000100:00100000:0.0:1788572517.436081:0:91550:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91550:1875449616271232:192.168.202.113@tcp:41:lfs.0 00800000:00000004:0.0:1788572517.436125:0:91550:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: update ffff9ada44819e18 cache blocks 317392/353529 objects 1023736/1024000 00000080:00200080:0.0:1788572517.436209:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436239:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436258:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436263:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436268:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436271:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436276:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436279:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436283:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436286:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436290:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436293:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436298:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436302:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436306:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436309:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436312:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436315:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436319:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436323:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436327:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436331:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436334:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436338:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436342:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436346:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436349:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436353:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436357:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436360:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436364:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436367:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436370:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436374:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436377:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436380:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436383:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436386:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436389:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436392:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436395:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436399:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436402:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436405:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436409:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436411:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436414:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436417:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436421:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436549:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436558:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436564:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436568:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436572:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436575:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436578:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436582:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436585:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436589:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436592:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436596:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436599:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436603:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436606:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436672:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436678:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436682:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436685:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436688:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436692:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436695:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436699:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436703:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436707:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436710:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436714:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436717:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436720:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436724:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436727:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436731:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436734:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436738:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436741:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436745:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436748:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436751:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436755:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436758:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436761:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436765:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436768:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436771:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436774:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436778:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436782:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436786:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436789:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436794:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436797:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436801:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436804:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436808:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436812:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436816:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436819:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436823:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436826:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436830:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436833:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436838:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436841:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.436846:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.436849:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.437083:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.437091:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.437096:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.437099:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.437103:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.437107:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.437110:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.437114:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.437117:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.437122:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.437126:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00800000:00000080:0.0:1788572517.437129:0:91550:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.437134:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.437140:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00020000:00000004:0.0:1788572517.437149:0:91550:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: age 4426, max_age 4434 00000100:00100000:0.0:1788572517.437196:0:91550:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91550:1875449616271360:192.168.202.113@tcp:13:lfs.0 00000100:00100000:0.0:1788572517.438076:0:91550:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada604fe3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572517.448801:0:91550:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91550:1875449616271360:192.168.202.113@tcp:13:lfs.0 00020000:00000004:0.0:1788572517.448831:0:91550:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: update ffff9ada4354d198 cache blocks 901356/958279 objects 261728/262144 00000080:00200080:0.0:1788572517.448943:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.448962:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00020000:00000004:0.0:1788572517.448982:0:91550:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: age 4426, max_age 4434 00000100:00100000:0.0:1788572517.449016:0:91550:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91550:1875449616271488:192.168.202.113@tcp:13:lfs.0 00000100:00100000:0.0:1788572517.449094:0:91550:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada604fe3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788572517.482913:0:91550:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91550:1875449616271488:192.168.202.113@tcp:13:lfs.0 00020000:00000004:0.0:1788572517.482951:0:91550:0:(obd_class.h:1116:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: update ffff9ada43549e18 cache blocks 901363/958279 objects 261734/262144 00000080:00200080:0.0:1788572517.483039:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483060:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483068:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483072:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483076:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483080:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483084:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483088:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483092:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483095:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483110:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483114:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483119:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483122:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483126:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483130:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483134:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483137:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483140:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483143:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483147:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483150:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483154:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483158:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483161:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483165:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483169:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483172:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483176:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483179:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483184:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483197:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483202:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483206:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483210:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483213:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483217:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483220:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483224:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483227:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483231:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483235:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483239:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483254:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483260:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483264:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483268:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483272:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483276:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483279:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483283:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483287:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483291:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483294:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483299:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483302:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483306:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483311:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483315:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483318:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483322:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483327:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483331:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483335:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483338:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483341:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483345:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483348:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483352:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483355:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483360:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483363:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483367:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483370:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483374:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483378:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483382:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483386:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483390:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483394:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483398:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483401:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483406:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483409:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483412:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.483416:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.483420:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.484948:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.484989:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.485006:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.485012:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.485017:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.485022:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.485025:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.485029:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.485032:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.485037:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.485040:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.485044:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.485047:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.485052:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.485055:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.485059:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.485062:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.485066:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.485070:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.485073:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.485078:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.485082:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.485085:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.485089:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.485092:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.485159:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.485166:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.485172:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.485176:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.485179:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.485235:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.485251:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.485257:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.485261:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.485265:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.485269:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.485272:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200080:0.0:1788572517.485276:0:91550:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffc7b743de0 00020000:00000080:0.0:1788572517.485280:0:91550:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571400 uarg=0000000000000000 00000080:00200008:0.0:1788572517.485296:0:91550:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00010000:00010000:1.0:1788572518.265306:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572518.776801:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449616271616:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572518.776819:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616271744:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572518.777219:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616271872:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572518.783759:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616272000:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572518.785176:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616272128:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572518.801316:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616271872:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572518.801344:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616272128:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572518.801416:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616271744:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572518.801440:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616272000:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572518.802493:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449616271616:192.168.202.113@tcp:400:kworker.0 00010000:00010000:0.0:1788572519.289638:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:2.0:1788572519.747887:0:91568:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=40086685 len=0 uarg=00007ffd4a9d2d20 00000002:00000080:2.0:1788572519.747909:0:91568:0:(mdc_request.c:2317:mdc_iocontrol()) lustre-MDT0000-mdc-ffff9ada58b0a000: cmd=40086685 len=576 karg=ffff9ada45668c00 uarg=0000000000000000 00000100:00080000:2.0:1788572519.747919:0:91568:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:2.0:1788572519.747929:0:91568:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff9ada58b0a000: Connection to lustre-MDT0000 (at 192.168.202.113@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:2.0:1788572519.747936:0:91568:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572519.747942:0:91568:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4429 00000100:00080000:2.0:1788572519.747972:0:91568:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572519.747985:0:91568:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788572519.748018:0:91568:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616272256/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788572519.748044:0:91568:0:(recover.c:337:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery started, waiting 20 seconds 00000100:00100000:1.0:1788572519.749258:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616272256:192.168.202.113@tcp:38:lctl.0 00010000:00010000:3.0:1788572520.313084:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572521.336695:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572522.363413:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572523.384246:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788572523.895662:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572523.895670:0:58077:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1788572523.895740:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616272512:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572523.896025:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616272768:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572523.896072:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1788572523.896080:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449616272384:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572523.896114:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616272640:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572523.903297:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449616272384:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572523.903323:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616272768:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572523.905027:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616272640:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572523.915970:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616272512:192.168.202.113@tcp:400:kworker.0 00010000:00010000:1.0:1788572524.407795:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788572525.041004:0:2382:0:(import.c:1176:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to target with instance 52 00000100:00080000:1.0:1788572525.041053:0:2382:0:(import.c:1341:ptlrpc_connect_interpret()) reconnected to lustre-MDT0000_UUID@192.168.202.113@tcp after partition 00000100:00080000:1.0:1788572525.041058:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to RECOVER 00000100:00080000:1.0:1788572525.041065:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from RECOVER to FULL 00000100:02000000:1.0:1788572525.041400:0:2382:0:(import.c:1787:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff9ada58b0a000: Connection restored to 192.168.202.113@tcp (at 192.168.202.113@tcp) 00000100:00100000:1.0:1788572525.041421:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616272256:192.168.202.113@tcp:38:lctl.0 00000100:00080000:3.0:1788572525.043482:0:91568:0:(recover.c:348:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery finished FULL, rc = 0 00000080:00200008:0.0:1788572525.097300:0:91571:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1788572525.097317:0:91571:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572525.097342:0:91571:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572525.097362:0:91571:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 15 lru_type: 1 00010000:00010000:0.0:1788572525.097384:0:91571:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 15 lru_type: 1 00000080:00200000:0.0:1788572525.097393:0:91571:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572525.097401:0:91571:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572525.097405:0:91571:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:0.0:1788572525.097412:0:91571:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1e6fcb8 00010000:00010000:0.0:1788572525.097416:0:91571:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 15 lru_type: 1 00010000:00010000:0.0:1788572525.097426:0:91571:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 16 lru_type: 1 00000080:00200008:0.0:1788572525.099071:0:91571:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1788572525.099081:0:91571:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572525.099166:0:91571:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572525.099183:0:91571:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 16 lru_type: 1 00010000:00010000:0.0:1788572525.099196:0:91571:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 16 lru_type: 1 00000080:00200000:0.0:1788572525.099203:0:91571:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572525.099210:0:91571:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572525.099214:0:91571:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:0.0:1788572525.099220:0:91571:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1e6fcb8 00010000:00010000:0.0:1788572525.099224:0:91571:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 16 lru_type: 1 00010000:00010000:0.0:1788572525.099233:0:91571:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 17 lru_type: 1 00000080:00200000:0.0:1788572525.100401:0:91571:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572525.100424:0:91571:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572525.100438:0:91571:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 17 lru_type: 1 00010000:00010000:0.0:1788572525.100451:0:91571:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 17 lru_type: 1 00000080:00200000:0.0:1788572525.100458:0:91571:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572525.100464:0:91571:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572525.100467:0:91571:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:0.0:1788572525.100473:0:91571:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1e6fbb0 00010000:00010000:0.0:1788572525.100476:0:91571:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 17 lru_type: 1 00010000:00010000:0.0:1788572525.100485:0:91571:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 18 lru_type: 1 00000080:00200000:0.0:1788572525.100496:0:91571:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:0.0:1788572525.100505:0:91571:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200080:0.0:1788572525.100608:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:0.0:1788572525.100624:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00800000:00000004:0.0:1788572525.100631:0:91571:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: age 4435, max_age 4441 00000100:00100000:0.0:1788572525.100672:0:91571:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91571:1875449616272896:192.168.202.113@tcp:41:lfs.0 00000100:00100000:0.0:1788572525.102113:0:91571:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada604fe3c0 going to sleep for 12 seconds 00000100:00100000:0.0:1788572525.107198:0:91571:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91571:1875449616272896:192.168.202.113@tcp:41:lfs.0 00800000:00000004:0.0:1788572525.107229:0:91571:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: update ffff9ada4481d198 cache blocks 317325/353529 objects 1023687/1024000 00000080:00200080:3.0:1788572525.107363:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:3.0:1788572525.107382:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45535000 uarg=0000000000000000 00800000:00000004:3.0:1788572525.107390:0:91571:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: age 4435, max_age 4441 00000100:00100000:3.0:1788572525.107440:0:91571:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91571:1875449616273024:192.168.202.113@tcp:41:lfs.0 00000100:00100000:3.0:1788572525.107610:0:91571:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a29c0 going to sleep for 12 seconds 00000100:00100000:1.0:1788572525.109638:0:91571:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91571:1875449616273024:192.168.202.113@tcp:41:lfs.0 00800000:00000004:1.0:1788572525.109664:0:91571:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: update ffff9ada44819e18 cache blocks 317392/353529 objects 1023736/1024000 00000080:00200080:1.0:1788572525.109728:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.109746:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.109753:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.109758:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.109762:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.109765:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.109770:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.109773:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.109777:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.109780:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.109785:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.109789:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.109792:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.109796:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.109799:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.109802:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.109805:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.109808:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.109812:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.109815:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.109818:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.109822:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.109825:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.109828:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.109832:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.109836:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.109839:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.109843:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.109847:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.109850:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.109890:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.109895:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.109899:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.109902:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.109906:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.109910:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.109914:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.109917:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.109921:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.109924:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.109928:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.109931:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.109945:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.109950:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.109955:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.109957:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.109961:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.109964:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.109968:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.109971:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.109975:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.109978:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.109982:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.109985:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.109989:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.109993:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.109996:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.110000:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.110004:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.110008:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.110012:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.110016:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.110020:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.110024:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.110028:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.110032:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.110035:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.110039:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.110043:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.110046:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.110050:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.110053:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.110057:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.110060:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.110065:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.110068:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.110072:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.110075:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.110079:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.110082:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.110086:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.110090:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.110094:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.110098:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.110102:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.110106:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.110110:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.110113:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.110118:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.110121:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.110125:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.110128:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.110132:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.110135:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.110139:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.110142:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.110146:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.110149:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.110153:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.110156:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.110160:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.110163:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.110167:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.110171:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.110175:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.110178:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.110193:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.110198:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.110202:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.110206:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.110211:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.110214:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.110218:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.110222:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.110225:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.110228:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.110232:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.110235:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.110239:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.110242:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.110246:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.110249:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.110253:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.110257:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.110261:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00800000:00000080:1.0:1788572525.110264:0:91571:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.110268:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.110275:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00020000:00000004:1.0:1788572525.110283:0:91571:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: age 4435, max_age 4441 00000100:00100000:1.0:1788572525.110326:0:91571:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91571:1875449616273152:192.168.202.113@tcp:13:lfs.0 00000100:00100000:1.0:1788572525.110392:0:91571:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada50a4f900 going to sleep for 11 seconds 00000100:00100000:1.0:1788572525.112759:0:91571:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91571:1875449616273152:192.168.202.113@tcp:13:lfs.0 00020000:00000004:1.0:1788572525.112786:0:91571:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: update ffff9ada4354d198 cache blocks 901356/958279 objects 261728/262144 00000080:00200080:1.0:1788572525.112887:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.113079:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00020000:00000004:1.0:1788572525.113090:0:91571:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: age 4435, max_age 4441 00000100:00100000:1.0:1788572525.113125:0:91571:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91571:1875449616273280:192.168.202.113@tcp:13:lfs.0 00000100:00100000:1.0:1788572525.113197:0:91571:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada50a4f900 going to sleep for 11 seconds 00000100:00100000:1.0:1788572525.117200:0:91571:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91571:1875449616273280:192.168.202.113@tcp:13:lfs.0 00020000:00000004:1.0:1788572525.117231:0:91571:0:(obd_class.h:1116:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: update ffff9ada43549e18 cache blocks 901363/958279 objects 261734/262144 00000080:00200080:1.0:1788572525.117303:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.117323:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.117330:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.117335:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.117339:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.117343:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.117348:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.117351:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.117355:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.117359:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.117363:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.117367:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.117371:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.117375:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.117379:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.117383:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.117387:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.117390:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.117397:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.117401:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.117405:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.117409:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.117413:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.117416:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.117420:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.117423:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.117752:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.117763:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.117771:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.117775:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.117779:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.117782:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.117786:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.117789:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.117793:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.117796:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.117800:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.117803:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.117807:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.117811:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.117815:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.117819:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.117822:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.117826:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.117830:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.117833:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.117837:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.117840:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.117843:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.117846:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.117850:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.117896:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.117903:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.117907:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.117911:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.117914:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.117918:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.117922:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.117925:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.117929:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.117933:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.117946:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.117951:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.117955:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.117959:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.117962:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.117966:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.117969:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.117974:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.117977:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.117981:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.117984:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.117988:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.117991:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.117995:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.117998:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.118002:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.118005:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.118009:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.118013:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.118017:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.118020:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.118024:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.118027:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.118031:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.118035:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.118039:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.118043:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.118047:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.118051:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.118055:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.118058:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.118061:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.118065:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.118068:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.118071:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.118074:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.118077:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.118081:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.118084:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.118087:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.118090:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.118093:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.118096:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.118099:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.118103:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.118106:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.118109:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.118113:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.118116:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.118120:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.118124:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.118127:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.118131:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.118135:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.118140:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.118143:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.118146:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.118150:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.118154:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.118158:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.118161:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.118165:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.118169:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200080:1.0:1788572525.118172:0:91571:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe28ba2540 00020000:00000080:1.0:1788572525.118176:0:91571:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada5164b800 uarg=0000000000000000 00000080:00200008:1.0:1788572525.118321:0:91571:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00010000:00010000:3.0:1788572525.431723:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572526.455054:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:3.0:1788572527.163522:0:91589:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=40086685 len=0 uarg=00007ffef630d370 00000002:00000080:3.0:1788572527.163544:0:91589:0:(mdc_request.c:2317:mdc_iocontrol()) lustre-MDT0000-mdc-ffff9ada58b0a000: cmd=40086685 len=576 karg=ffff9ada45536800 uarg=0000000000000000 00000100:00080000:3.0:1788572527.163552:0:91589:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:3.0:1788572527.163561:0:91589:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff9ada58b0a000: Connection to lustre-MDT0000 (at 192.168.202.113@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:3.0:1788572527.163567:0:91589:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572527.163574:0:91589:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4437 00000100:00080000:3.0:1788572527.163589:0:91589:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572527.163674:0:91589:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572527.163783:0:91589:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3800 x1875449616273408/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572527.163818:0:91589:0:(recover.c:337:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery started, waiting 20 seconds 00000100:00100000:0.0:1788572527.164591:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616273408:192.168.202.113@tcp:38:lctl.0 00010000:00010000:0.0:1788572527.479432:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572528.503097:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788572529.015530:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572529.015537:0:85951:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1788572529.015654:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449616273536:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572529.015764:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616273792:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572529.015798:0:2383:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1788572529.015805:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616273664:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572529.015827:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616273920:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572529.027589:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616273664:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572529.027608:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616273920:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572529.027619:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449616273536:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572529.027630:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616273792:192.168.202.113@tcp:400:kworker.0 00010000:00010000:3.0:1788572529.527279:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788572530.039107:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572530.039114:0:85951:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1788572530.039157:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616274176:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572530.039258:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616274432:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572530.039290:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1788572530.039296:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449616274048:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572530.039325:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616274304:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572530.046809:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616274176:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572530.049474:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449616274048:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572530.060700:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616274304:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572530.060722:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616274432:192.168.202.113@tcp:400:kworker.0 00010000:00010000:0.0:1788572530.551630:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572531.575095:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788572532.226205:0:2382:0:(import.c:1176:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to target with instance 52 00000100:00080000:0.0:1788572532.226242:0:2382:0:(import.c:1341:ptlrpc_connect_interpret()) reconnected to lustre-MDT0000_UUID@192.168.202.113@tcp after partition 00000100:00080000:0.0:1788572532.226246:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to RECOVER 00000100:00080000:0.0:1788572532.226252:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from RECOVER to FULL 00000100:02000000:0.0:1788572532.226818:0:2382:0:(import.c:1787:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff9ada58b0a000: Connection restored to 192.168.202.113@tcp (at 192.168.202.113@tcp) 00000100:00100000:0.0:1788572532.226836:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616273408:192.168.202.113@tcp:38:lctl.0 00000100:00080000:1.0:1788572532.227146:0:91589:0:(recover.c:348:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery finished FULL, rc = 0 00000080:00200008:3.0:1788572532.267358:0:91592:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1788572532.267373:0:91592:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:3.0:1788572532.267399:0:91592:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1788572532.267420:0:91592:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 18 lru_type: 1 00010000:00010000:3.0:1788572532.267437:0:91592:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 18 lru_type: 1 00000080:00200000:3.0:1788572532.267445:0:91592:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1788572532.267452:0:91592:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:3.0:1788572532.267456:0:91592:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:3.0:1788572532.267463:0:91592:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1ed7cb8 00010000:00010000:3.0:1788572532.267467:0:91592:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 18 lru_type: 1 00010000:00010000:3.0:1788572532.267477:0:91592:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 19 lru_type: 1 00000080:00200008:3.0:1788572532.268040:0:91592:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 0, flags 0, foreign 0 00000080:00200000:3.0:1788572532.268046:0:91592:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:3.0:1788572532.268055:0:91592:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1788572532.268063:0:91592:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 19 lru_type: 1 00010000:00010000:3.0:1788572532.268073:0:91592:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 19 lru_type: 1 00000080:00200000:3.0:1788572532.268080:0:91592:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1788572532.268084:0:91592:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:3.0:1788572532.268086:0:91592:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:3.0:1788572532.268090:0:91592:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1ed7cb8 00010000:00010000:3.0:1788572532.268092:0:91592:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 19 lru_type: 1 00010000:00010000:3.0:1788572532.268099:0:91592:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 20 lru_type: 1 00000080:00200000:3.0:1788572532.272114:0:91592:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:3.0:1788572532.272147:0:91592:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1788572532.272166:0:91592:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 20 lru_type: 1 00010000:00010000:3.0:1788572532.272183:0:91592:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 20 lru_type: 1 00000080:00200000:3.0:1788572532.272198:0:91592:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1788572532.272205:0:91592:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:3.0:1788572532.272208:0:91592:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:3.0:1788572532.272215:0:91592:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1ed7bb0 00010000:00010000:3.0:1788572532.272219:0:91592:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 20 lru_type: 1 00010000:00010000:3.0:1788572532.272228:0:91592:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 21 lru_type: 1 00000080:00200000:3.0:1788572532.272331:0:91592:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:3.0:1788572532.272341:0:91592:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200080:3.0:1788572532.272547:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:3.0:1788572532.272563:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45534800 uarg=0000000000000000 00800000:00000004:3.0:1788572532.272569:0:91592:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: age 4442, max_age 4448 00000100:00100000:3.0:1788572532.272612:0:91592:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91592:1875449616274560:192.168.202.113@tcp:41:lfs.0 00000100:00100000:3.0:1788572532.273074:0:91592:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2240 going to sleep for 12 seconds 00000100:00100000:0.0:1788572532.280975:0:91592:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91592:1875449616274560:192.168.202.113@tcp:41:lfs.0 00800000:00000004:0.0:1788572532.281002:0:91592:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: update ffff9ada4481d198 cache blocks 317325/353529 objects 1023687/1024000 00000080:00200080:0.0:1788572532.281099:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.281116:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00800000:00000004:0.0:1788572532.281124:0:91592:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: age 4442, max_age 4448 00000100:00100000:0.0:1788572532.282005:0:91592:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91592:1875449616274688:192.168.202.113@tcp:41:lfs.0 00000100:00100000:0.0:1788572532.282091:0:91592:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada604fe9c0 going to sleep for 12 seconds 00000100:00100000:0.0:1788572532.295440:0:91592:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91592:1875449616274688:192.168.202.113@tcp:41:lfs.0 00800000:00000004:0.0:1788572532.295471:0:91592:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: update ffff9ada44819e18 cache blocks 317392/353529 objects 1023736/1024000 00000080:00200080:0.0:1788572532.295554:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295571:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295579:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295583:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295587:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295592:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295595:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295599:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295603:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295606:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295610:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295613:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295617:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295620:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295623:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295626:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295629:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295633:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295637:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295640:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295644:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295647:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295651:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295654:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295657:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295660:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295678:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295683:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295687:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295690:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295694:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295697:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295700:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295704:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295707:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295710:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295714:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295717:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295722:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295725:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295729:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295732:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295736:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295740:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295744:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295747:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295751:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295755:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295759:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295762:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295766:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295770:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295773:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295777:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295780:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295784:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295788:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295791:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295795:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295799:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295803:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295806:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295810:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295814:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295818:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295821:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295825:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295828:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295831:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295834:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295837:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295840:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295844:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295847:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295851:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295896:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295902:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295906:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295921:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295925:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295929:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295933:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295936:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295940:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295944:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295948:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295952:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295955:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295959:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295963:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295967:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295971:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295975:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295979:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295983:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295986:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295990:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.295993:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.295997:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.296001:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.296005:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.296009:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.296013:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.296016:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.296021:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.296025:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.296028:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.296032:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:0.0:1788572532.296036:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:0.0:1788572532.296112:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45572800 uarg=0000000000000000 00000080:00200080:1.0:1788572532.296258:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:1.0:1788572532.296327:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.296336:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:1.0:1788572532.296339:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.296343:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:1.0:1788572532.296347:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.296351:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:1.0:1788572532.296354:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.296357:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:1.0:1788572532.296360:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.296363:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:1.0:1788572532.296366:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.296370:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:1.0:1788572532.296374:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.296378:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00800000:00000080:1.0:1788572532.296381:0:91592:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.296385:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.296390:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00020000:00000004:1.0:1788572532.296398:0:91592:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: age 4442, max_age 4448 00000100:00100000:1.0:1788572532.296510:0:91592:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91592:1875449616274816:192.168.202.113@tcp:13:lfs.0 00000100:00100000:1.0:1788572532.296655:0:91592:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada50a4f240 going to sleep for 11 seconds 00000100:00100000:1.0:1788572532.302453:0:91592:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91592:1875449616274816:192.168.202.113@tcp:13:lfs.0 00020000:00000004:1.0:1788572532.302487:0:91592:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: update ffff9ada4354d198 cache blocks 901356/958279 objects 261728/262144 00000080:00200080:1.0:1788572532.302564:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.302591:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00020000:00000004:1.0:1788572532.302599:0:91592:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: age 4442, max_age 4448 00000100:00100000:1.0:1788572532.303004:0:91592:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91592:1875449616274944:192.168.202.113@tcp:13:lfs.0 00000100:00100000:1.0:1788572532.303081:0:91592:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada50a4f240 going to sleep for 11 seconds 00000100:00100000:1.0:1788572532.312652:0:91592:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91592:1875449616274944:192.168.202.113@tcp:13:lfs.0 00020000:00000004:1.0:1788572532.312688:0:91592:0:(obd_class.h:1116:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: update ffff9ada43549e18 cache blocks 901363/958279 objects 261734/262144 00000080:00200080:1.0:1788572532.312849:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.312886:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.312895:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.312899:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.312903:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.312917:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.312923:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.312927:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.312932:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.312935:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.312939:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.312943:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.312947:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.312950:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.312955:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.312958:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.312962:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.312965:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.312969:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.312972:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.312979:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.312983:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.312987:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.312991:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.312995:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.312998:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313002:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313005:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313009:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313013:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313017:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313020:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313024:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313027:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313030:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313033:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313037:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313040:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313044:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313047:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313050:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313053:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313057:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313060:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313063:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313067:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313071:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313074:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313078:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313081:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313085:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313088:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313092:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313096:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313100:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313103:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313107:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313110:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313113:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313117:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313121:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313124:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313128:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313131:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313135:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313138:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313141:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313144:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313148:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313151:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313156:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313159:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313163:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313167:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313171:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313175:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313178:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313182:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313185:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313188:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313192:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313195:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313199:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313202:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313206:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313210:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313213:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313216:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313220:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313223:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313227:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313230:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313233:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313236:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313240:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313243:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313247:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313251:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313254:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313259:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313262:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313266:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313270:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313273:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313277:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313280:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313283:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313286:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313290:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313293:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313297:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313300:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313304:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313307:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313311:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313314:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313318:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313322:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313325:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313329:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313333:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313337:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313340:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313344:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200080:1.0:1788572532.313348:0:91592:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffebeee7070 00020000:00000080:1.0:1788572532.313352:0:91592:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51648400 uarg=0000000000000000 00000080:00200008:1.0:1788572532.313384:0:91592:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00010000:00010000:3.0:1788572532.600849:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572533.624372:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572534.647340:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:1.0:1788572534.730368:0:91611:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=40086685 len=0 uarg=00007fffc85f55f0 00000002:00000080:1.0:1788572534.730397:0:91611:0:(mdc_request.c:2317:mdc_iocontrol()) lustre-MDT0000-mdc-ffff9ada58b0a000: cmd=40086685 len=576 karg=ffff9ada51648400 uarg=0000000000000000 00000100:00080000:1.0:1788572534.730408:0:91611:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:1.0:1788572534.730419:0:91611:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff9ada58b0a000: Connection to lustre-MDT0000 (at 192.168.202.113@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:1.0:1788572534.730427:0:91611:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572534.730432:0:91611:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4444 00000100:00080000:1.0:1788572534.730451:0:91611:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572534.730465:0:91611:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572534.730505:0:91611:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d180 x1875449616275072/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788572534.730667:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616275072:192.168.202.113@tcp:38:lctl.0 00000100:00080000:1.0:1788572534.731462:0:91611:0:(recover.c:337:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery started, waiting 20 seconds 00000100:00080000:3.0:1788572535.160319:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572535.160328:0:58077:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1788572535.160447:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449616275200:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572535.160591:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616275456:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572535.160635:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1788572535.160642:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616275328:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572535.160671:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616275584:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572535.167206:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616275456:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572535.167244:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616275328:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572535.167256:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616275584:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572535.167267:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449616275200:192.168.202.113@tcp:400:kworker.0 00010000:00010000:3.0:1788572535.671218:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572536.695317:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788572537.207332:0:58077:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572537.207343:0:58077:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1788572537.207471:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616275840:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572537.207601:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616276096:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572537.207640:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1788572537.207647:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449616275712:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572537.207679:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616275968:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572537.210712:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449616275712:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572537.210749:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616275840:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572537.210763:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616276096:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572537.217163:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616275968:192.168.202.113@tcp:400:kworker.0 00010000:00010000:3.0:1788572537.719468:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572538.744325:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572539.767296:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788572539.870769:0:2382:0:(import.c:1176:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to target with instance 52 00000100:00080000:0.0:1788572539.870814:0:2382:0:(import.c:1341:ptlrpc_connect_interpret()) reconnected to lustre-MDT0000_UUID@192.168.202.113@tcp after partition 00000100:00080000:0.0:1788572539.870819:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to RECOVER 00000100:00080000:0.0:1788572539.870844:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from RECOVER to FULL 00000100:02000000:0.0:1788572539.871271:0:2382:0:(import.c:1787:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff9ada58b0a000: Connection restored to 192.168.202.113@tcp (at 192.168.202.113@tcp) 00000100:00100000:0.0:1788572539.871289:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616275072:192.168.202.113@tcp:38:lctl.0 00000100:00080000:1.0:1788572539.871375:0:91611:0:(recover.c:348:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery finished FULL, rc = 0 00000080:00200008:2.0:1788572539.899911:0:91613:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1788572539.899928:0:91613:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788572539.899954:0:91613:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788572539.899974:0:91613:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 21 lru_type: 1 00010000:00010000:2.0:1788572539.899988:0:91613:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 21 lru_type: 1 00000080:00200000:2.0:1788572539.899994:0:91613:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788572539.900003:0:91613:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788572539.900007:0:91613:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:2.0:1788572539.900014:0:91613:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1e67cb8 00010000:00010000:2.0:1788572539.900017:0:91613:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 21 lru_type: 1 00010000:00010000:2.0:1788572539.900024:0:91613:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 22 lru_type: 1 00000080:00200008:2.0:1788572539.900530:0:91613:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 0, flags 0, foreign 0 00000080:00200000:2.0:1788572539.900533:0:91613:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788572539.900542:0:91613:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788572539.900549:0:91613:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 22 lru_type: 1 00010000:00010000:2.0:1788572539.900555:0:91613:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 22 lru_type: 1 00000080:00200000:2.0:1788572539.900560:0:91613:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788572539.900563:0:91613:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788572539.900565:0:91613:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:2.0:1788572539.900568:0:91613:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1e67cb8 00010000:00010000:2.0:1788572539.900570:0:91613:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 22 lru_type: 1 00010000:00010000:2.0:1788572539.900575:0:91613:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 23 lru_type: 1 00000080:00200000:2.0:1788572539.900660:0:91613:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788572539.900666:0:91613:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788572539.900671:0:91613:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 23 lru_type: 1 00010000:00010000:2.0:1788572539.900677:0:91613:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 23 lru_type: 1 00000080:00200000:2.0:1788572539.900682:0:91613:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788572539.900684:0:91613:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788572539.900686:0:91613:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:2.0:1788572539.900689:0:91613:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1e67bb0 00010000:00010000:2.0:1788572539.900691:0:91613:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 23 lru_type: 1 00010000:00010000:2.0:1788572539.900696:0:91613:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 24 lru_type: 1 00000080:00200000:2.0:1788572539.900700:0:91613:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:2.0:1788572539.900706:0:91613:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200080:2.0:1788572539.901712:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:2.0:1788572539.901735:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669000 uarg=0000000000000000 00800000:00000004:2.0:1788572539.901744:0:91613:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: age 4449, max_age 4456 00000100:00100000:2.0:1788572539.901798:0:91613:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91613:1875449616276224:192.168.202.113@tcp:41:lfs.0 00000100:00100000:2.0:1788572539.902096:0:91613:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6076cb40 going to sleep for 12 seconds 00000100:00100000:2.0:1788572539.906963:0:91613:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91613:1875449616276224:192.168.202.113@tcp:41:lfs.0 00800000:00000004:2.0:1788572539.907003:0:91613:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: update ffff9ada4481d198 cache blocks 317325/353529 objects 1023687/1024000 00000080:00200080:2.0:1788572539.907150:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.907241:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00800000:00000004:0.0:1788572539.907253:0:91613:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: age 4449, max_age 4456 00000100:00100000:0.0:1788572539.907301:0:91613:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91613:1875449616276352:192.168.202.113@tcp:41:lfs.0 00000100:00100000:0.0:1788572539.907403:0:91613:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada604fee40 going to sleep for 12 seconds 00000100:00100000:0.0:1788572539.910756:0:91613:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91613:1875449616276352:192.168.202.113@tcp:41:lfs.0 00800000:00000004:0.0:1788572539.910794:0:91613:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: update ffff9ada44819e18 cache blocks 317392/353529 objects 1023736/1024000 00000080:00200080:0.0:1788572539.912108:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912149:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912165:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912170:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912175:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912180:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912184:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912188:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912192:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912196:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912199:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912202:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912206:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912209:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912213:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912217:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912221:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912224:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912229:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912233:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912237:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912245:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912250:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912254:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912258:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912261:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912265:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912269:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912273:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912276:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912280:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912283:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912287:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912290:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912295:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912298:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912302:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912305:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912308:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912311:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912315:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912318:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912322:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912325:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912329:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912333:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912337:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912341:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912344:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912348:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912352:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912355:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912359:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912363:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912367:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912372:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912377:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912381:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912385:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912388:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912392:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912395:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912399:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912402:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912406:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912410:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912413:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912418:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912424:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912427:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912430:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912434:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912437:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912441:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912445:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912448:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912452:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912456:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912460:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912463:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912468:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912471:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912476:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912493:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912505:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912511:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912515:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912519:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912523:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912526:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912531:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912535:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912539:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912544:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912548:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912552:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912556:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912559:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912563:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912566:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912570:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912574:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912578:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912582:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912586:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912590:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912593:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912597:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912601:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912605:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912608:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912612:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912615:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912618:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912621:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912624:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912628:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912631:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912634:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912637:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912642:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912645:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912648:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912652:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912655:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00800000:00000080:0.0:1788572539.912659:0:91613:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.912664:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.912671:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00020000:00000004:0.0:1788572539.912686:0:91613:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: age 4449, max_age 4456 00000100:00100000:0.0:1788572539.912756:0:91613:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91613:1875449616276480:192.168.202.113@tcp:13:lfs.0 00000100:00100000:0.0:1788572539.912916:0:91613:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada604fee40 going to sleep for 11 seconds 00000100:00100000:0.0:1788572539.918421:0:91613:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91613:1875449616276480:192.168.202.113@tcp:13:lfs.0 00020000:00000004:0.0:1788572539.918459:0:91613:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: update ffff9ada4354d198 cache blocks 901356/958279 objects 261728/262144 00000080:00200080:0.0:1788572539.918638:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.918661:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00020000:00000004:0.0:1788572539.918669:0:91613:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: age 4449, max_age 4456 00000100:00100000:0.0:1788572539.918708:0:91613:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91613:1875449616276608:192.168.202.113@tcp:13:lfs.0 00000100:00100000:0.0:1788572539.918829:0:91613:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada604fee40 going to sleep for 11 seconds 00000100:00100000:0.0:1788572539.924099:0:91613:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91613:1875449616276608:192.168.202.113@tcp:13:lfs.0 00020000:00000004:0.0:1788572539.924151:0:91613:0:(obd_class.h:1116:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: update ffff9ada43549e18 cache blocks 901363/958279 objects 261734/262144 00000080:00200080:0.0:1788572539.924250:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.924275:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.924284:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.924289:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.924293:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.924297:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.924301:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.924305:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.924309:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.924313:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.924316:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.924320:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.924323:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.924327:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.924331:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.924334:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.924338:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.924342:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.924346:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.924350:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.924353:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.924356:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.924360:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.924363:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.924366:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.924369:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.924372:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.924375:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.924380:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.924383:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.924388:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.924391:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.924395:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.924398:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.924402:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.924406:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.924410:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.924413:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.924417:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.924421:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.924426:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.924429:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.924434:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.924437:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.924442:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.924445:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.924449:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.924453:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.924455:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.924458:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.924460:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.924462:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.924464:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.924466:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.924468:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.924470:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.924472:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.924474:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.924476:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.924477:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.924479:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.924481:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.924483:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.924484:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.924489:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.924493:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.925354:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.925373:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.925382:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.925386:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.925391:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.925395:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.925399:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.925402:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.925406:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.925409:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.925414:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.925417:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.925421:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.925425:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.925429:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.925433:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.925437:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.925441:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.925444:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.925447:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.925450:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.925452:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.925454:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.925457:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.925459:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.925461:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.925463:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.925465:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.925467:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.925469:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.925471:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.925472:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.925474:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.925476:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.925478:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.925481:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.925484:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.925487:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.926096:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.926112:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.926124:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.926128:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.926132:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.926135:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.926139:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.926142:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.926146:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.926149:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.926153:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.926156:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.926160:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.926164:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.926168:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.926171:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.926175:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.926179:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.926183:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.926187:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200080:0.0:1788572539.926191:0:91613:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe36141bc0 00020000:00000080:0.0:1788572539.926194:0:91613:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45573c00 uarg=0000000000000000 00000080:00200008:0.0:1788572539.926213:0:91613:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00010000:00010000:3.0:1788572540.792459:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572541.815511:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:2.0:1788572542.106721:0:91632:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=40086685 len=0 uarg=00007ffef558eb80 00000002:00000080:2.0:1788572542.106743:0:91632:0:(mdc_request.c:2317:mdc_iocontrol()) lustre-MDT0000-mdc-ffff9ada58b0a000: cmd=40086685 len=576 karg=ffff9ada4566ac00 uarg=0000000000000000 00000100:00080000:2.0:1788572542.109702:0:91632:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:2.0:1788572542.109715:0:91632:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff9ada58b0a000: Connection to lustre-MDT0000 (at 192.168.202.113@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:2.0:1788572542.109721:0:91632:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572542.109727:0:91632:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4452 00000100:00080000:2.0:1788572542.109742:0:91632:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572542.109821:0:91632:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788572542.109900:0:91632:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616276736/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788572542.109925:0:91632:0:(recover.c:337:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery started, waiting 20 seconds 00000100:00100000:0.0:1788572542.110141:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616276736:192.168.202.113@tcp:38:lctl.0 00000100:00080000:0.0:1788572542.327134:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572542.327142:0:85951:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1788572542.327189:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616276992:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572542.327283:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616277248:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572542.327316:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1788572542.327323:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449616276864:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572542.327351:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616277120:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572542.331189:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616277120:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572542.331206:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616276992:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572542.331216:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616277248:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572542.332440:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449616276864:192.168.202.113@tcp:400:kworker.0 00010000:00010000:3.0:1788572542.841185:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572543.868823:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572544.887420:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788572545.337305:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572545.337314:0:85951:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1788572545.337371:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616277504:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572545.337481:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616277760:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572545.337518:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1788572545.337524:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449616277376:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572545.337558:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616277632:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572545.342455:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449616277376:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572545.342506:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616277632:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572545.342523:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616277504:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572545.343122:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616277760:192.168.202.113@tcp:400:kworker.0 00010000:00010000:3.0:1788572545.915646:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572546.935108:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788572547.601146:0:2382:0:(import.c:1176:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to target with instance 52 00000100:00080000:0.0:1788572547.601198:0:2382:0:(import.c:1341:ptlrpc_connect_interpret()) reconnected to lustre-MDT0000_UUID@192.168.202.113@tcp after partition 00000100:00080000:0.0:1788572547.601203:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to RECOVER 00000100:00080000:0.0:1788572547.601212:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from RECOVER to FULL 00000100:02000000:0.0:1788572547.603362:0:2382:0:(import.c:1787:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff9ada58b0a000: Connection restored to 192.168.202.113@tcp (at 192.168.202.113@tcp) 00000100:00080000:2.0:1788572547.612028:0:91632:0:(recover.c:348:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery finished FULL, rc = 0 00000100:00100000:0.0:1788572547.617991:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616276736:192.168.202.113@tcp:38:lctl.0 00000080:00200008:0.0:1788572547.635691:0:91634:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1788572547.635714:0:91634:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572547.635753:0:91634:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572547.635784:0:91634:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 24 lru_type: 1 00010000:00010000:0.0:1788572547.635806:0:91634:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 24 lru_type: 1 00000080:00200000:0.0:1788572547.635816:0:91634:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572547.635826:0:91634:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572547.635830:0:91634:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:0.0:1788572547.635842:0:91634:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1e7fcb8 00010000:00010000:0.0:1788572547.635884:0:91634:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 24 lru_type: 1 00010000:00010000:0.0:1788572547.635896:0:91634:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 25 lru_type: 1 00000080:00200008:0.0:1788572547.636641:0:91634:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1788572547.636648:0:91634:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572547.636665:0:91634:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572547.636676:0:91634:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 25 lru_type: 1 00010000:00010000:0.0:1788572547.636687:0:91634:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 25 lru_type: 1 00000080:00200000:0.0:1788572547.636693:0:91634:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572547.636699:0:91634:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572547.636702:0:91634:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:0.0:1788572547.636707:0:91634:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1e7fcb8 00010000:00010000:0.0:1788572547.636710:0:91634:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 25 lru_type: 1 00010000:00010000:0.0:1788572547.636718:0:91634:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 26 lru_type: 1 00000080:00200000:0.0:1788572547.636844:0:91634:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572547.646060:0:91634:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572547.646088:0:91634:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 26 lru_type: 1 00010000:00010000:0.0:1788572547.646108:0:91634:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 26 lru_type: 1 00000080:00200000:0.0:1788572547.646118:0:91634:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572547.646132:0:91634:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572547.646137:0:91634:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:0.0:1788572547.646147:0:91634:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1e7fbb0 00010000:00010000:0.0:1788572547.646152:0:91634:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 26 lru_type: 1 00010000:00010000:0.0:1788572547.646162:0:91634:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 27 lru_type: 1 00000080:00200000:0.0:1788572547.646181:0:91634:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:0.0:1788572547.646198:0:91634:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200080:0.0:1788572547.646397:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:0.0:1788572547.646417:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571800 uarg=0000000000000000 00800000:00000004:0.0:1788572547.646427:0:91634:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: age 4457, max_age 4464 00000100:00100000:0.0:1788572547.648793:0:91634:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91634:1875449616277888:192.168.202.113@tcp:41:lfs.0 00000100:00100000:0.0:1788572547.649175:0:91634:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada604fe9c0 going to sleep for 12 seconds 00000100:00100000:0.0:1788572547.665073:0:91634:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91634:1875449616277888:192.168.202.113@tcp:41:lfs.0 00800000:00000004:0.0:1788572547.665123:0:91634:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: update ffff9ada4481d198 cache blocks 317325/353529 objects 1023687/1024000 00000080:00200080:3.0:1788572547.677117:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:3.0:1788572547.677159:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45537c00 uarg=0000000000000000 00800000:00000004:3.0:1788572547.677173:0:91634:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: age 4457, max_age 4464 00000100:00100000:3.0:1788572547.677240:0:91634:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91634:1875449616278016:192.168.202.113@tcp:41:lfs.0 00000100:00100000:3.0:1788572547.677537:0:91634:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a26c0 going to sleep for 12 seconds 00000100:00100000:1.0:1788572547.686373:0:91634:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91634:1875449616278016:192.168.202.113@tcp:41:lfs.0 00800000:00000004:1.0:1788572547.686431:0:91634:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: update ffff9ada44819e18 cache blocks 317392/353529 objects 1023736/1024000 00000080:00200080:1.0:1788572547.686538:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686562:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686573:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686579:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686583:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686587:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686591:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686595:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686599:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686603:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686607:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686610:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686614:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686617:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686620:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686623:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686627:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686631:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686634:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686638:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686642:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686646:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686650:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686654:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686658:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686661:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686665:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686668:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686673:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686675:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686679:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686683:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686687:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686690:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686693:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686696:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686701:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686704:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686708:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686712:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686716:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686719:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686735:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686740:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686744:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686747:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686751:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686755:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686758:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686762:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686767:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686770:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686774:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686779:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686782:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686786:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686790:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686793:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686797:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686802:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686806:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686810:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686814:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686817:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686821:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686824:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686827:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686830:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686833:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686837:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686841:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686844:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686913:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686918:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686922:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686925:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686930:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686933:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686938:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686942:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686946:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686950:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686954:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686958:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686962:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686966:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686970:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686973:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686978:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686981:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686986:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686989:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.686994:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.686997:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.687001:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.687005:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.687009:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.687013:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.687017:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.687021:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.687025:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.687028:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.687032:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.687037:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.687040:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.687044:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.687047:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.687051:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.687055:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.687058:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.687062:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.687065:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.687069:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.687072:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.687076:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.687079:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.687083:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.687086:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.687090:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.687093:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.687097:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.687100:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.687104:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.687108:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.687111:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00800000:00000080:1.0:1788572547.687115:0:91634:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.687119:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.687126:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00020000:00000004:1.0:1788572547.687137:0:91634:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: age 4457, max_age 4464 00000100:00100000:1.0:1788572547.687190:0:91634:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91634:1875449616278144:192.168.202.113@tcp:13:lfs.0 00000100:00100000:1.0:1788572547.687293:0:91634:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada50a4fa80 going to sleep for 11 seconds 00000100:00100000:1.0:1788572547.698708:0:91634:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91634:1875449616278144:192.168.202.113@tcp:13:lfs.0 00020000:00000004:1.0:1788572547.698761:0:91634:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: update ffff9ada4354d198 cache blocks 901356/958279 objects 261728/262144 00000080:00200080:1.0:1788572547.699103:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.699130:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00020000:00000004:1.0:1788572547.699140:0:91634:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: age 4457, max_age 4464 00000100:00100000:1.0:1788572547.699189:0:91634:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91634:1875449616278272:192.168.202.113@tcp:13:lfs.0 00000100:00100000:1.0:1788572547.699295:0:91634:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada50a4fa80 going to sleep for 11 seconds 00000100:00100000:1.0:1788572547.719810:0:91634:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91634:1875449616278272:192.168.202.113@tcp:13:lfs.0 00020000:00000004:1.0:1788572547.719901:0:91634:0:(obd_class.h:1116:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: update ffff9ada43549e18 cache blocks 901363/958279 objects 261734/262144 00000080:00200080:1.0:1788572547.719992:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720014:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720023:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720028:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720033:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720037:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720041:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720045:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720049:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720052:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720056:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720059:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720064:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720067:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720071:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720074:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720077:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720080:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720084:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720087:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720090:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720092:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720096:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720098:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720102:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720105:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720109:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720112:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720116:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720119:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720123:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720126:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720130:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720133:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720137:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720141:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720145:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720150:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720153:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720156:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720160:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720163:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720167:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720170:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720173:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720176:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720179:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720182:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720185:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720188:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720191:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720194:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720197:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720199:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720202:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720205:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720208:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720210:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720213:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720216:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720219:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720222:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720228:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720231:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720235:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720238:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720242:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720245:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720249:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720252:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720255:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720258:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720262:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720264:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720267:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720270:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720273:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720276:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720279:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720283:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720287:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720291:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720295:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720298:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720302:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720306:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720310:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720314:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.720318:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.720321:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.721005:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.721018:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.721023:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.721027:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.721031:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.721035:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.721040:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.721044:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.721048:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.721052:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.721055:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.721059:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.721063:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.721067:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.721071:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.721075:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.721080:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.721083:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.721087:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.721090:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.721093:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.721096:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.721101:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.721105:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.721109:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.721112:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.721116:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.721120:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.721125:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.721128:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.721132:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.721135:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.721139:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.721143:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572547.721147:0:91634:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffec599a740 00020000:00000080:1.0:1788572547.721151:0:91634:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200008:1.0:1788572547.721172:0:91634:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00010000:00010000:0.0:1788572547.962084:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572548.983388:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:2.0:1788572549.875522:0:91653:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=40086685 len=0 uarg=00007ffe8f4afc00 00000002:00000080:2.0:1788572549.875550:0:91653:0:(mdc_request.c:2317:mdc_iocontrol()) lustre-MDT0000-mdc-ffff9ada58b0a000: cmd=40086685 len=576 karg=ffff9ada4566a000 uarg=0000000000000000 00000100:00080000:2.0:1788572549.875562:0:91653:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:2.0:1788572549.875573:0:91653:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff9ada58b0a000: Connection to lustre-MDT0000 (at 192.168.202.113@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:2.0:1788572549.875582:0:91653:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572549.875587:0:91653:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4459 00000100:00080000:2.0:1788572549.875608:0:91653:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572549.875622:0:91653:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788572549.875664:0:91653:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616278400/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788572549.875804:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616278400:192.168.202.113@tcp:38:lctl.0 00000100:00080000:2.0:1788572549.876062:0:91653:0:(recover.c:337:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery started, waiting 20 seconds 00010000:00010000:3.0:1788572550.008133:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788572550.520358:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572550.520368:0:91222:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1788572550.523850:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449616278528:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572550.524075:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616278784:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572550.524120:0:2383:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1788572550.524129:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616278656:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572550.524160:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616278912:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572550.528494:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616278912:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572550.528515:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616278784:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572550.533496:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616278656:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572550.533520:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449616278528:192.168.202.113@tcp:400:kworker.0 00010000:00010000:0.0:1788572551.033969:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572552.065675:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788572552.568742:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572552.568751:0:91222:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1788572552.568850:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616279168:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572552.569019:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616279424:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572552.569062:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1788572552.569068:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449616279040:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572552.569102:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616279296:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572552.573538:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616279296:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572552.573562:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616279168:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572552.573573:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616279424:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572552.574050:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449616279040:192.168.202.113@tcp:400:kworker.0 00010000:00010000:3.0:1788572553.080100:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572554.103919:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572555.127634:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788572555.256693:0:2382:0:(import.c:1176:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to target with instance 52 00000100:00080000:0.0:1788572555.256749:0:2382:0:(import.c:1341:ptlrpc_connect_interpret()) reconnected to lustre-MDT0000_UUID@192.168.202.113@tcp after partition 00000100:00080000:0.0:1788572555.256754:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to RECOVER 00000100:00080000:0.0:1788572555.256762:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from RECOVER to FULL 00000100:02000000:1.0:1788572555.259931:0:2382:0:(import.c:1787:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff9ada58b0a000: Connection restored to 192.168.202.113@tcp (at 192.168.202.113@tcp) 00000100:00100000:1.0:1788572555.259961:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616278400:192.168.202.113@tcp:38:lctl.0 00000100:00080000:2.0:1788572555.260252:0:91653:0:(recover.c:348:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery finished FULL, rc = 0 00000080:00200008:3.0:1788572555.294439:0:91655:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1788572555.294458:0:91655:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:3.0:1788572555.294489:0:91655:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1788572555.294515:0:91655:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 27 lru_type: 1 00010000:00010000:3.0:1788572555.294534:0:91655:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 27 lru_type: 1 00000080:00200000:3.0:1788572555.294544:0:91655:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1788572555.294554:0:91655:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:3.0:1788572555.294559:0:91655:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:3.0:1788572555.294568:0:91655:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1f67cb8 00010000:00010000:3.0:1788572555.294573:0:91655:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 27 lru_type: 1 00010000:00010000:3.0:1788572555.294584:0:91655:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 28 lru_type: 1 00000080:00200008:3.0:1788572555.298704:0:91655:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 0, flags 0, foreign 0 00000080:00200000:3.0:1788572555.298717:0:91655:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:3.0:1788572555.298758:0:91655:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1788572555.298783:0:91655:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 28 lru_type: 1 00010000:00010000:3.0:1788572555.299582:0:91655:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 28 lru_type: 1 00000080:00200000:3.0:1788572555.299594:0:91655:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1788572555.299605:0:91655:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:3.0:1788572555.299609:0:91655:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:3.0:1788572555.299620:0:91655:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1f67cb8 00010000:00010000:3.0:1788572555.299624:0:91655:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 28 lru_type: 1 00010000:00010000:3.0:1788572555.299635:0:91655:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 29 lru_type: 1 00000080:00200000:3.0:1788572555.301504:0:91655:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:3.0:1788572555.301543:0:91655:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1788572555.301566:0:91655:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 29 lru_type: 1 00010000:00010000:3.0:1788572555.301586:0:91655:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 29 lru_type: 1 00000080:00200000:3.0:1788572555.301600:0:91655:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1788572555.301607:0:91655:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:3.0:1788572555.301611:0:91655:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:3.0:1788572555.301619:0:91655:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1f67bb0 00010000:00010000:3.0:1788572555.301624:0:91655:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 29 lru_type: 1 00010000:00010000:3.0:1788572555.301633:0:91655:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 30 lru_type: 1 00000080:00200000:3.0:1788572555.301647:0:91655:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:3.0:1788572555.301660:0:91655:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200080:3.0:1788572555.301813:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:3.0:1788572555.301833:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45537400 uarg=0000000000000000 00800000:00000004:3.0:1788572555.301842:0:91655:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: age 4465, max_age 4471 00000100:00100000:3.0:1788572555.301931:0:91655:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91655:1875449616279552:192.168.202.113@tcp:41:lfs.0 00000100:00100000:3.0:1788572555.302424:0:91655:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2480 going to sleep for 12 seconds 00000100:00100000:0.0:1788572555.308115:0:91655:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91655:1875449616279552:192.168.202.113@tcp:41:lfs.0 00800000:00000004:0.0:1788572555.308157:0:91655:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: update ffff9ada4481d198 cache blocks 317325/353529 objects 1023687/1024000 00000080:00200080:1.0:1788572555.317191:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.317233:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00800000:00000004:1.0:1788572555.317245:0:91655:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: age 4465, max_age 4471 00000100:00100000:1.0:1788572555.317309:0:91655:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91655:1875449616279680:192.168.202.113@tcp:41:lfs.0 00000100:00100000:1.0:1788572555.317438:0:91655:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada50a4ff00 going to sleep for 12 seconds 00000100:00100000:1.0:1788572555.321024:0:91655:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91655:1875449616279680:192.168.202.113@tcp:41:lfs.0 00800000:00000004:1.0:1788572555.321069:0:91655:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: update ffff9ada44819e18 cache blocks 317392/353529 objects 1023736/1024000 00000080:00200080:1.0:1788572555.321160:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321183:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321192:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321196:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321200:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321202:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321205:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321207:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321209:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321211:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321213:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321214:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321216:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321218:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321220:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321222:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321225:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321227:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321229:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321230:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321232:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321234:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321236:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321238:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321240:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321241:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321243:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321245:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321247:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321248:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321251:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321252:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321254:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321256:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321258:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321259:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321261:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321263:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321265:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321266:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321268:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321271:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321273:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321274:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321276:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321278:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321280:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321282:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321283:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321285:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321287:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321289:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321291:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321292:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321294:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321296:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321298:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321299:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321301:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321303:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321305:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321307:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321309:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321310:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321312:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321314:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321316:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321318:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321320:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321322:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321324:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321325:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321327:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321329:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321331:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321332:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321334:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321549:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321559:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321564:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321568:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321571:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321575:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321578:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321582:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321585:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321589:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321593:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321597:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321600:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321604:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321609:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321613:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321616:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321620:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321625:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321629:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321632:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321636:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321640:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321644:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321648:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321652:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321656:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321658:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321660:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321663:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321664:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321666:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321668:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321670:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321672:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321674:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321675:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321677:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321679:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321682:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321684:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321687:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321688:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321691:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321692:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321694:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321696:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321698:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00800000:00000080:1.0:1788572555.321699:0:91655:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.321702:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.321706:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00020000:00000004:1.0:1788572555.321714:0:91655:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: age 4465, max_age 4471 00000100:00100000:1.0:1788572555.321767:0:91655:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91655:1875449616279808:192.168.202.113@tcp:13:lfs.0 00000100:00100000:1.0:1788572555.326230:0:91655:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada50a4ff00 going to sleep for 11 seconds 00000100:00100000:1.0:1788572555.335436:0:91655:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91655:1875449616279808:192.168.202.113@tcp:13:lfs.0 00020000:00000004:1.0:1788572555.335477:0:91655:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: update ffff9ada4354d198 cache blocks 901356/958279 objects 261728/262144 00000080:00200080:1.0:1788572555.335623:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.335647:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00020000:00000004:1.0:1788572555.335657:0:91655:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: age 4465, max_age 4472 00000100:00100000:1.0:1788572555.335700:0:91655:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91655:1875449616279936:192.168.202.113@tcp:13:lfs.0 00000100:00100000:1.0:1788572555.335806:0:91655:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada50a4ff00 going to sleep for 11 seconds 00000100:00100000:1.0:1788572555.338738:0:91655:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91655:1875449616279936:192.168.202.113@tcp:13:lfs.0 00020000:00000004:1.0:1788572555.338773:0:91655:0:(obd_class.h:1116:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: update ffff9ada43549e18 cache blocks 901363/958279 objects 261734/262144 00000080:00200080:1.0:1788572555.339466:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339496:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339505:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339509:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339514:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339517:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339521:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339524:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339528:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339532:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339536:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339539:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339543:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339547:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339551:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339555:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339558:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339562:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339566:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339570:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339575:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339579:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339582:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339586:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339590:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339594:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339598:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339602:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339605:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339608:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339612:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339615:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339618:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339621:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339626:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339629:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339632:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339636:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339639:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339643:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339646:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339649:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339653:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339656:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339660:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339664:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339668:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339671:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339675:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339678:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339683:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339686:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339689:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339693:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339696:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339699:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339703:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339706:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339711:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339714:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339730:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339735:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339739:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339742:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339746:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339750:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339754:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339758:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339762:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339765:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339770:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339773:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339777:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339781:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339785:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339790:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339794:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339798:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339802:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339805:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339809:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339814:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339818:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339821:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339825:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339828:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339833:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339836:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339840:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339844:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339848:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339879:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339885:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339890:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339894:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339897:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339901:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339905:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339909:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339913:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339918:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339921:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339925:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339927:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339931:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339934:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339938:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339942:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339945:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339948:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339952:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339955:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339959:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339963:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339977:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339982:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339988:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339991:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.339994:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.339998:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.340002:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.340005:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.340009:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.340128:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200080:1.0:1788572555.340138:0:91655:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffd1359fa70 00020000:00000080:1.0:1788572555.340143:0:91655:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649400 uarg=0000000000000000 00000080:00200008:1.0:1788572555.340164:0:91655:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00010000:00010000:3.0:1788572556.152221:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572557.175331:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:0.0:1788572557.451425:0:91674:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=40086685 len=0 uarg=00007fffd7ecad50 00000002:00000080:0.0:1788572557.451454:0:91674:0:(mdc_request.c:2317:mdc_iocontrol()) lustre-MDT0000-mdc-ffff9ada58b0a000: cmd=40086685 len=576 karg=ffff9ada45571400 uarg=0000000000000000 00000100:00080000:0.0:1788572557.451465:0:91674:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:0.0:1788572557.451476:0:91674:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff9ada58b0a000: Connection to lustre-MDT0000 (at 192.168.202.113@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:0.0:1788572557.451485:0:91674:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572557.451490:0:91674:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4467 00000100:00080000:0.0:1788572557.451509:0:91674:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788572557.451524:0:91674:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788572557.451564:0:91674:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616280064/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788572557.451601:0:91674:0:(recover.c:337:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery started, waiting 20 seconds 00000100:00100000:0.0:1788572557.451672:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616280064:192.168.202.113@tcp:38:lctl.0 00000100:00080000:3.0:1788572557.687369:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572557.687378:0:77466:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1788572557.687445:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616280320:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572557.687564:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616280576:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572557.687599:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1788572557.687606:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449616280192:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572557.687663:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616280448:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572557.698276:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449616280192:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572557.698309:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616280448:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572557.698325:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616280576:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572557.703669:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616280320:192.168.202.113@tcp:400:kworker.0 00010000:00010000:3.0:1788572558.200642:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572559.224120:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788572559.737532:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572559.737541:0:77466:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1788572559.738127:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449616280704:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572559.738275:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616280960:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572559.738316:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1788572559.738322:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616280832:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572559.738348:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616281088:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572559.758159:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616280832:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572559.758196:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616281088:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572559.758210:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449616280704:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572559.758221:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616280960:192.168.202.113@tcp:400:kworker.0 00010000:00010000:0.0:1788572560.249263:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572561.275702:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572562.295652:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788572562.923692:0:2382:0:(import.c:1176:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to target with instance 52 00000100:00080000:0.0:1788572562.923737:0:2382:0:(import.c:1341:ptlrpc_connect_interpret()) reconnected to lustre-MDT0000_UUID@192.168.202.113@tcp after partition 00000100:00080000:0.0:1788572562.923742:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to RECOVER 00000100:00080000:0.0:1788572562.923751:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from RECOVER to FULL 00000100:02000000:2.0:1788572562.925587:0:2382:0:(import.c:1787:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff9ada58b0a000: Connection restored to 192.168.202.113@tcp (at 192.168.202.113@tcp) 00000100:00100000:2.0:1788572562.925618:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616280064:192.168.202.113@tcp:38:lctl.0 00000100:00080000:1.0:1788572562.925820:0:91674:0:(recover.c:348:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery finished FULL, rc = 0 00000080:00200008:2.0:1788572562.959450:0:91676:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1788572562.960155:0:91676:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788572562.960194:0:91676:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788572562.960220:0:91676:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 30 lru_type: 1 00010000:00010000:2.0:1788572562.960239:0:91676:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 30 lru_type: 1 00000080:00200000:2.0:1788572562.960249:0:91676:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788572562.960261:0:91676:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788572562.960265:0:91676:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:2.0:1788572562.960275:0:91676:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1ebfcb8 00010000:00010000:2.0:1788572562.960279:0:91676:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 30 lru_type: 1 00010000:00010000:2.0:1788572562.960289:0:91676:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 31 lru_type: 1 00000080:00200008:2.0:1788572562.968057:0:91676:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 0, flags 0, foreign 0 00000080:00200000:2.0:1788572562.968080:0:91676:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788572562.968121:0:91676:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788572562.968160:0:91676:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 31 lru_type: 1 00010000:00010000:2.0:1788572562.968180:0:91676:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 31 lru_type: 1 00000080:00200000:2.0:1788572562.968190:0:91676:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788572562.968200:0:91676:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788572562.968205:0:91676:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:2.0:1788572562.968214:0:91676:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1ebfcb8 00010000:00010000:2.0:1788572562.968219:0:91676:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 31 lru_type: 1 00010000:00010000:2.0:1788572562.968229:0:91676:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 32 lru_type: 1 00000080:00200000:2.0:1788572562.968436:0:91676:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788572562.968447:0:91676:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788572562.968455:0:91676:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 32 lru_type: 1 00010000:00010000:2.0:1788572562.968464:0:91676:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 32 lru_type: 1 00000080:00200000:2.0:1788572562.968471:0:91676:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788572562.968476:0:91676:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788572562.968478:0:91676:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:2.0:1788572562.968482:0:91676:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1ebfbb0 00010000:00010000:2.0:1788572562.968486:0:91676:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 32 lru_type: 1 00010000:00010000:2.0:1788572562.968494:0:91676:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 33 lru_type: 1 00000080:00200000:2.0:1788572562.968503:0:91676:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:2.0:1788572562.968514:0:91676:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200080:2.0:1788572562.968631:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:2.0:1788572562.968650:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45669400 uarg=0000000000000000 00800000:00000004:2.0:1788572562.968659:0:91676:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: age 4472, max_age 4479 00000100:00100000:2.0:1788572562.968718:0:91676:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91676:1875449616281216:192.168.202.113@tcp:41:lfs.0 00000100:00100000:2.0:1788572562.968846:0:91676:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6076ce40 going to sleep for 12 seconds 00000100:00100000:0.0:1788572562.978133:0:91676:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91676:1875449616281216:192.168.202.113@tcp:41:lfs.0 00800000:00000004:0.0:1788572562.978179:0:91676:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: update ffff9ada4481d198 cache blocks 317325/353529 objects 1023687/1024000 00000080:00200080:0.0:1788572562.978294:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.978319:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00800000:00000004:0.0:1788572562.978330:0:91676:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: age 4472, max_age 4479 00000100:00100000:0.0:1788572562.978382:0:91676:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91676:1875449616281344:192.168.202.113@tcp:41:lfs.0 00000100:00100000:0.0:1788572562.979326:0:91676:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada604fe600 going to sleep for 12 seconds 00000100:00100000:0.0:1788572562.982665:0:91676:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91676:1875449616281344:192.168.202.113@tcp:41:lfs.0 00800000:00000004:0.0:1788572562.982722:0:91676:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: update ffff9ada44819e18 cache blocks 317392/353529 objects 1023736/1024000 00000080:00200080:0.0:1788572562.982850:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.982919:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.982930:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.982935:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.982939:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.982943:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.982948:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.982951:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.982955:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.982959:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.982963:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.982967:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.982972:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.982975:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.982980:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.982984:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.982988:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.982992:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.982996:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983000:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983004:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983008:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983013:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983017:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983021:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983025:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983042:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983047:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983051:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983055:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983059:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983062:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983066:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983069:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983072:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983076:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983080:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983084:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983088:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983092:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983102:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983106:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983111:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983115:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983119:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983123:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983127:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983131:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983135:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983138:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983142:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983145:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983149:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983152:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983157:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983160:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983165:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983169:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983173:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983177:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983181:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983184:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983189:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983192:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983195:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983199:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983203:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983206:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983210:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983213:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983217:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983221:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983225:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983228:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983232:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983235:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983239:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983242:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983246:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983249:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983253:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983256:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983260:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983263:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983267:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983271:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983275:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983278:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983282:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983286:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983291:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983295:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983299:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983303:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983307:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983311:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983315:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983319:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983323:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983327:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983331:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983335:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983339:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983343:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983347:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983350:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983354:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983358:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983362:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983365:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983370:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983373:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983377:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983381:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983385:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983389:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983393:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983397:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983402:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983405:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983409:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983413:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983417:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983433:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983439:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00800000:00000080:0.0:1788572562.983444:0:91676:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572562.983448:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:0.0:1788572562.983456:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00020000:00000004:0.0:1788572562.983477:0:91676:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: age 4473, max_age 4479 00000100:00100000:0.0:1788572562.983532:0:91676:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91676:1875449616281472:192.168.202.113@tcp:13:lfs.0 00000100:00100000:0.0:1788572562.983650:0:91676:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada604fe600 going to sleep for 11 seconds 00000100:00100000:0.0:1788572562.996050:0:91676:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91676:1875449616281472:192.168.202.113@tcp:13:lfs.0 00020000:00000004:0.0:1788572562.996104:0:91676:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: update ffff9ada4354d198 cache blocks 901356/958279 objects 261728/262144 00000080:00200080:0.0:1788572562.996225:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:0.0:1788572562.996255:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00020000:00000004:0.0:1788572562.996266:0:91676:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: age 4473, max_age 4479 00000100:00100000:0.0:1788572562.996330:0:91676:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91676:1875449616281600:192.168.202.113@tcp:13:lfs.0 00000100:00100000:0.0:1788572562.996498:0:91676:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada604fe600 going to sleep for 11 seconds 00000100:00100000:0.0:1788572563.009390:0:91676:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91676:1875449616281600:192.168.202.113@tcp:13:lfs.0 00020000:00000004:0.0:1788572563.009446:0:91676:0:(obd_class.h:1116:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: update ffff9ada43549e18 cache blocks 901363/958279 objects 261734/262144 00000080:00200080:0.0:1788572563.009776:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:0.0:1788572563.009802:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572563.009811:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:0.0:1788572563.009815:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572563.009820:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:0.0:1788572563.009823:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572563.009827:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:0.0:1788572563.009830:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572563.009834:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:0.0:1788572563.009838:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572563.009842:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:0.0:1788572563.009845:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570c00 uarg=0000000000000000 00000080:00200080:0.0:1788572563.009850:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.013306:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.013352:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.013365:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.013370:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.013374:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.013379:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.013382:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.013386:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.013390:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.013394:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.013397:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.013401:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.013404:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.013407:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.013411:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.013415:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.013452:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.016208:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.016249:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.016261:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.016266:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.016270:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.016274:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.016278:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.016282:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.016285:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.016289:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.016292:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.016295:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.016299:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.016303:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.016307:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.016310:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.016314:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.016317:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.016321:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.016324:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.016329:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.016332:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.016336:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.016339:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.016343:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.016346:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.016349:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.016352:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.016356:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.016359:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.016363:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.016366:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.016371:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.016374:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.016378:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.016382:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.016386:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.016390:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.016394:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.016398:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.016402:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.016405:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.016409:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.016413:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.016417:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.016432:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.016437:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.016441:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.016447:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.016450:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.016454:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.016927:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.016944:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.016954:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.016958:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.016962:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.016965:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.016969:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.016972:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.016975:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.016979:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.016981:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.016984:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.016987:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.016990:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.016992:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.016996:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.016998:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.017001:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.017004:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.017007:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.017010:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.017013:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.017016:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.017019:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.017022:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.017025:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.017039:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.017043:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.017047:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.017052:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.017055:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.017059:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.017062:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.017066:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.017069:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.017072:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.017076:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.017080:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.017083:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.017087:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.017090:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.017095:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.017098:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572563.017103:0:91676:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffca9d799d0 00020000:00000080:1.0:1788572563.017106:0:91676:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200008:1.0:1788572563.017126:0:91676:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00010000:00010000:0.0:1788572563.319306:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572564.345253:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572564.862410:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449616281728:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572564.879825:0:2383:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1788572564.879876:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616281856:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572564.882823:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449616281728:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572564.883645:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616282112:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572564.883753:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1788572564.883761:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616281984:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572564.883791:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616282240:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572564.896510:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616281856:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572564.896701:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616281984:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572564.902454:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616282240:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572564.902750:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616282112:192.168.202.113@tcp:400:kworker.0 00000020:00000080:1.0:1788572565.169087:0:91695:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=40086685 len=0 uarg=00007ffee7b946b0 00000002:00000080:1.0:1788572565.169124:0:91695:0:(mdc_request.c:2317:mdc_iocontrol()) lustre-MDT0000-mdc-ffff9ada58b0a000: cmd=40086685 len=576 karg=ffff9ada51649000 uarg=0000000000000000 00000100:00080000:1.0:1788572565.169136:0:91695:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:1.0:1788572565.169147:0:91695:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff9ada58b0a000: Connection to lustre-MDT0000 (at 192.168.202.113@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:1.0:1788572565.169155:0:91695:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572565.169161:0:91695:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4475 00000100:00080000:1.0:1788572565.169182:0:91695:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572565.169197:0:91695:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572565.169236:0:91695:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e680 x1875449616282368/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572565.169274:0:91695:0:(recover.c:337:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery started, waiting 20 seconds 00000100:00100000:0.0:1788572565.169518:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616282368:192.168.202.113@tcp:38:lctl.0 00010000:00010000:3.0:1788572565.367603:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572566.392612:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572567.417639:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572568.439189:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572569.463197:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788572569.975291:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572569.975301:0:91222:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1788572569.975640:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449616282496:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572569.975808:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616282752:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572569.975892:0:2384:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1788572569.975901:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616282624:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572569.975935:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616282880:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572569.985494:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616282624:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572569.985518:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616282880:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572569.985529:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449616282496:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572569.985540:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616282752:192.168.202.113@tcp:400:kworker.0 00010000:00010000:3.0:1788572570.493808:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788572570.620459:0:2382:0:(import.c:1176:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to target with instance 52 00000100:00080000:0.0:1788572570.620502:0:2382:0:(import.c:1341:ptlrpc_connect_interpret()) reconnected to lustre-MDT0000_UUID@192.168.202.113@tcp after partition 00000100:00080000:0.0:1788572570.620507:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to RECOVER 00000100:00080000:0.0:1788572570.620515:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from RECOVER to FULL 00000100:02000000:2.0:1788572570.622272:0:2382:0:(import.c:1787:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff9ada58b0a000: Connection restored to 192.168.202.113@tcp (at 192.168.202.113@tcp) 00000100:00100000:2.0:1788572570.622302:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616282368:192.168.202.113@tcp:38:lctl.0 00000100:00080000:1.0:1788572570.622450:0:91695:0:(recover.c:348:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery finished FULL, rc = 0 00000080:00200008:1.0:1788572570.659086:0:91698:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1788572570.659106:0:91698:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:1.0:1788572570.664401:0:91698:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1788572570.665542:0:91698:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 33 lru_type: 1 00010000:00010000:1.0:1788572570.665566:0:91698:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 33 lru_type: 1 00000080:00200000:1.0:1788572570.665583:0:91698:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1788572570.665598:0:91698:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:1.0:1788572570.665602:0:91698:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:1.0:1788572570.665613:0:91698:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1d4fcb8 00010000:00010000:1.0:1788572570.665627:0:91698:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 33 lru_type: 1 00010000:00010000:1.0:1788572570.665638:0:91698:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 34 lru_type: 1 00000080:00200008:1.0:1788572570.666529:0:91698:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 0, flags 0, foreign 0 00000080:00200000:1.0:1788572570.666539:0:91698:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:1.0:1788572570.666562:0:91698:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1788572570.666582:0:91698:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 34 lru_type: 1 00010000:00010000:1.0:1788572570.666597:0:91698:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 34 lru_type: 1 00000080:00200000:1.0:1788572570.666604:0:91698:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1788572570.666611:0:91698:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:1.0:1788572570.666614:0:91698:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:1.0:1788572570.666621:0:91698:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1d4fcb8 00010000:00010000:1.0:1788572570.666624:0:91698:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 34 lru_type: 1 00010000:00010000:1.0:1788572570.666643:0:91698:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 35 lru_type: 1 00000080:00200000:1.0:1788572570.666785:0:91698:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:1.0:1788572570.666795:0:91698:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1788572570.666802:0:91698:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 35 lru_type: 1 00010000:00010000:1.0:1788572570.666809:0:91698:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 35 lru_type: 1 00000080:00200000:1.0:1788572570.666816:0:91698:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1788572570.666820:0:91698:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:1.0:1788572570.666822:0:91698:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:1.0:1788572570.666826:0:91698:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1d4fbb0 00010000:00010000:1.0:1788572570.666829:0:91698:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 35 lru_type: 1 00010000:00010000:1.0:1788572570.666836:0:91698:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 36 lru_type: 1 00000080:00200000:1.0:1788572570.666843:0:91698:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:1.0:1788572570.666851:0:91698:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200080:1.0:1788572570.667015:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.667036:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00800000:00000004:1.0:1788572570.667044:0:91698:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: age 4480, max_age 4487 00000100:00100000:1.0:1788572570.667104:0:91698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91698:1875449616283008:192.168.202.113@tcp:41:lfs.0 00000100:00100000:1.0:1788572570.672976:0:91698:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada50a4f780 going to sleep for 12 seconds 00000100:00100000:1.0:1788572570.684511:0:91698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91698:1875449616283008:192.168.202.113@tcp:41:lfs.0 00800000:00000004:1.0:1788572570.684557:0:91698:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: update ffff9ada4481d198 cache blocks 317325/353529 objects 1023687/1024000 00000080:00200080:1.0:1788572570.684665:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.684687:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00800000:00000004:1.0:1788572570.684698:0:91698:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: age 4480, max_age 4487 00000100:00100000:1.0:1788572570.684757:0:91698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91698:1875449616283136:192.168.202.113@tcp:41:lfs.0 00000100:00100000:1.0:1788572570.684846:0:91698:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada50a4f780 going to sleep for 12 seconds 00000100:00100000:1.0:1788572570.688041:0:91698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91698:1875449616283136:192.168.202.113@tcp:41:lfs.0 00800000:00000004:1.0:1788572570.688081:0:91698:0:(obd_class.h:1116:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: update ffff9ada44819e18 cache blocks 317392/353529 objects 1023736/1024000 00000080:00200080:1.0:1788572570.688185:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688208:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688218:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688223:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688228:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688232:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688237:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688240:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688244:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688249:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688253:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688257:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688261:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688266:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688270:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688274:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688278:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688282:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688287:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688291:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688296:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688300:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688304:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688307:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688311:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688315:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688319:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688322:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688326:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688329:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688334:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688338:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688341:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688344:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688348:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688351:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688355:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688359:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688365:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688369:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688373:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688377:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688380:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688383:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688387:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688389:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688393:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688395:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688398:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688400:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688403:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688540:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688550:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688556:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688560:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688563:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688567:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688570:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688574:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688577:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688581:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688584:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688589:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688593:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688596:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688600:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688604:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688607:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688611:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688615:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688618:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688622:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688626:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688629:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688634:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688637:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688641:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688644:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688648:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688652:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688655:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688658:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688662:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688666:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688670:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688674:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688677:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688681:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688686:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688689:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688693:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688696:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688700:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688704:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688708:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688712:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688715:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688719:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688723:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688726:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:1.0:1788572570.688731:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:1.0:1788572570.688734:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada51649000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.692558:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:0.0:1788572570.692602:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.692800:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:0.0:1788572570.692809:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.692815:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:0.0:1788572570.692819:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.692824:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:0.0:1788572570.692827:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.692832:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:0.0:1788572570.692836:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.692840:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:0.0:1788572570.692844:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.692848:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:0.0:1788572570.692851:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.692884:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:0.0:1788572570.692889:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.692892:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:0.0:1788572570.692895:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.692898:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:0.0:1788572570.692903:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.692907:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:0.0:1788572570.692911:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.692914:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00800000:00000080:0.0:1788572570.692918:0:91698:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.692922:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.692929:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00020000:00000004:0.0:1788572570.692940:0:91698:0:(obd_class.h:1090:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: age 4480, max_age 4487 00000100:00100000:0.0:1788572570.693005:0:91698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91698:1875449616283264:192.168.202.113@tcp:13:lfs.0 00000100:00100000:0.0:1788572570.695451:0:91698:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada604fed80 going to sleep for 11 seconds 00000100:00100000:0.0:1788572570.706953:0:91698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91698:1875449616283264:192.168.202.113@tcp:13:lfs.0 00020000:00000004:0.0:1788572570.706997:0:91698:0:(obd_class.h:1116:obd_statfs()) lustre-OST0000-osc-ffff9ada58b0a000: update ffff9ada4354d198 cache blocks 901356/958279 objects 261728/262144 00000080:00200080:0.0:1788572570.707110:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.707486:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00020000:00000004:0.0:1788572570.707499:0:91698:0:(obd_class.h:1090:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: age 4480, max_age 4487 00000100:00100000:0.0:1788572570.707553:0:91698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91698:1875449616283392:192.168.202.113@tcp:13:lfs.0 00000100:00100000:0.0:1788572570.707984:0:91698:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada604fed80 going to sleep for 11 seconds 00000100:00100000:0.0:1788572570.715256:0:91698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job lfs:35eb2d60-2de0-4b00-9815-d5153edd0057:91698:1875449616283392:192.168.202.113@tcp:13:lfs.0 00020000:00000004:0.0:1788572570.715298:0:91698:0:(obd_class.h:1116:obd_statfs()) lustre-OST0001-osc-ffff9ada58b0a000: update ffff9ada43549e18 cache blocks 901363/958279 objects 261734/262144 00000080:00200080:0.0:1788572570.715519:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.715545:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.715555:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.715560:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.715565:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.715569:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.715573:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.715577:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.715581:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.715586:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.715590:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.715593:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.715598:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.715603:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.715607:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.715609:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.715724:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.715733:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.715808:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.715816:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.715821:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.715825:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.715828:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.715832:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.715836:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.715839:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.715844:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.715847:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.715851:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.715908:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.715915:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.715918:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.715923:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.715927:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.715931:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.715935:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.715938:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.715942:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.715946:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.715949:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.715953:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.715957:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.715961:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.715965:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.715969:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.715973:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.715977:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.715981:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.715985:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.715989:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.715993:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.715997:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.716000:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.716003:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.716007:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.716010:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.716013:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.716016:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.716020:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.716023:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.716027:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.716031:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.716035:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.716039:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.716043:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.716046:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.716050:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.716054:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.716058:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.716061:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.716066:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.716069:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.716073:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.716077:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.716081:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.716085:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.716089:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.716092:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.716095:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.716098:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.716102:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.716105:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.716108:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.717308:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.717339:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.717349:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.717354:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.717358:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.717362:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.717711:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.717725:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.717733:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.717806:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.717816:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.717823:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.717827:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.717830:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.717834:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.717838:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.717842:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.717846:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.717849:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.717878:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.717883:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.717887:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.717890:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.717893:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.717896:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.717899:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.717903:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.717906:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.717910:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.717913:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.717916:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.717920:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.717923:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.717927:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.717930:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.717935:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.717939:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.717943:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.717947:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.717951:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.717955:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200080:0.0:1788572570.717959:0:91698:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffe61d7be20 00020000:00000080:0.0:1788572570.717963:0:91698:0:(lov_obd.c:976:lov_iocontrol()) lustre-clilov-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45570000 uarg=0000000000000000 00000080:00200008:0.0:1788572570.717982:0:91698:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00010000:00010000:3.0:1788572571.511179:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572572.541013:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572573.563176:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572574.598156:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572575.098244:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449616283520:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572575.098415:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616283776:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572575.098456:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616284032:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572575.098496:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1788572575.098503:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616283648:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572575.098530:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616283904:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572575.103050:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616283648:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572575.103068:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616283904:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572575.103077:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449616283520:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572575.103086:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616284032:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572575.107066:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616283776:192.168.202.113@tcp:400:kworker.0 00010000:00010000:2.0:1788572575.607052:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572576.633950:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572577.656159:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572578.683680:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572579.704066:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572580.216280:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449616284160:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572580.220808:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1788572580.220836:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616284288:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572580.235191:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616284544:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572580.235358:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616284672:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572580.240829:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616284416:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572580.246757:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616284288:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572580.246781:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449616284160:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572580.247618:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2384:1875449616284544:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572580.247656:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616284672:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572580.254290:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616284416:192.168.202.113@tcp:400:kworker.0 00010000:00010000:1.0:1788572580.730357:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1788572581.160932:0:92355:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == replay-single test 44b: race in target handle connect ========================================================== 21:42:59 (1788572579) 00010000:00010000:2.0:1788572581.757522:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572582.786040:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572583.799252:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:0.0:1788572584.276602:0:92391:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=40086685 len=0 uarg=00007fff254d30d0 00000002:00000080:0.0:1788572584.276635:0:92391:0:(mdc_request.c:2317:mdc_iocontrol()) lustre-MDT0000-mdc-ffff9ada58b0a000: cmd=40086685 len=576 karg=ffff9ada45572c00 uarg=0000000000000000 00000100:00080000:0.0:1788572584.276646:0:92391:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:0.0:1788572584.276657:0:92391:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff9ada58b0a000: Connection to lustre-MDT0000 (at 192.168.202.113@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:0.0:1788572584.276666:0:92391:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572584.276671:0:92391:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4482 00000100:00080000:0.0:1788572584.276690:0:92391:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788572584.276705:0:92391:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788572584.279110:0:92391:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d180 x1875449616284800/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788572584.279148:0:92391:0:(recover.c:337:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery started, waiting 20 seconds 00000100:00100000:2.0:1788572584.279275:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616284800:192.168.202.113@tcp:38:lctl.0 00010000:00010000:2.0:1788572584.823061:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788572585.339055:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572585.339068:0:85951:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1788572585.339136:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616285056:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572585.339260:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616285312:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572585.339300:0:2386:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1788572585.339307:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449616284928:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572585.339341:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616285184:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572585.348214:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2386:1875449616284928:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572585.348244:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616285184:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572585.348255:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616285056:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572585.348265:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616285312:192.168.202.113@tcp:400:kworker.0 00010000:00010000:3.0:1788572585.847253:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572586.871407:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572587.895111:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572588.921063:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572589.949224:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788572590.455466:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572590.455476:0:85951:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1788572590.455501:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada48407800 lustre-OST0000_UUID: changing import state from FULL to CONNECTING 00000100:00080000:3.0:1788572590.455506:0:85951:0:(import.c:2147:ptlrpc_disconnect_and_idle_import()) lustre-OST0000-osc-ffff9ada58b0a000: disconnect after 20s idle 00000100:00080000:3.0:1788572590.455531:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada48400800 lustre-OST0001_UUID: changing import state from FULL to CONNECTING 00000100:00080000:3.0:1788572590.455534:0:85951:0:(import.c:2147:ptlrpc_disconnect_and_idle_import()) lustre-OST0001-osc-ffff9ada58b0a000: disconnect after 20s idle 00000100:00100000:0.0:1788572590.455688:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616285696:192.168.202.113@tcp:9:kworker.0 00000100:00100000:0.0:1788572590.455831:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616285824:192.168.202.113@tcp:9:kworker.0 00000100:00100000:2.0:1788572590.455993:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449616285440:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572590.456040:0:2385:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1788572590.456047:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616285568:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572590.459762:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2385:1875449616285568:192.168.202.113@tcp:400:kworker.0 00000100:00080000:0.0:1788572590.459897:0:2382:0:(import.c:2057:ptlrpc_disconnect_idle_interpret()) @@@ inflight=1, refcount=3: rc = 0 req@ffff9ada604e0e00 x1875449616285696/t0(0) o9->lustre-OST0000-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572600 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572590.459911:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada48407800 lustre-OST0000_UUID: changing import state from CONNECTING to IDLE 00000100:00100000:0.0:1788572590.459914:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616285696:192.168.202.113@tcp:9:kworker.0 00000100:00080000:0.0:1788572590.459929:0:2382:0:(import.c:2057:ptlrpc_disconnect_idle_interpret()) @@@ inflight=1, refcount=3: rc = 0 req@ffff9ada604e1500 x1875449616285824/t0(0) o9->lustre-OST0001-osc-ffff9ada58b0a000@192.168.202.113@tcp:28/4 lens 224/224 e 0 to 0 dl 1788572600 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572590.459938:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada48400800 lustre-OST0001_UUID: changing import state from CONNECTING to IDLE 00000100:00100000:0.0:1788572590.459942:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616285824:192.168.202.113@tcp:9:kworker.0 00000100:00100000:2.0:1788572590.460619:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449616285440:192.168.202.113@tcp:400:kworker.0 00010000:00010000:2.0:1788572590.967089:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572592.004433:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572593.017349:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572594.039019:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788572594.550995:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788572584/real 1788572584] req@ffff9ada4557d180 x1875449616284800/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572594 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'lctl.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788572594.551029:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d180 x1875449616284800/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572594 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'lctl.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788572594.551049:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788572594.551083:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788572594.551102:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616284800:192.168.202.113@tcp:38:lctl.0 00000100:00080000:1.0:1788572594.551348:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572594.551352:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788572594.551355:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572594.551361:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4501 00000100:00080000:1.0:1788572594.551370:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 5s 00000100:00080000:1.0:1788572594.551376:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572594.551386:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572594.551399:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d880 x1875449616286080/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788572594.551445:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616286208:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572594.551595:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616286080:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788572594.551655:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449616285952:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572594.571473:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449616285952:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572594.573747:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616286208:192.168.202.113@tcp:400:kworker.0 00000100:00080000:2.0:1788572594.581675:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788572594.581689:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-114) 00000100:00100000:2.0:1788572594.581712:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616286080:192.168.202.113@tcp:38:kworker.0 00010000:00010000:3.0:1788572595.063983:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788572595.575389:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572595.575406:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788572595.575411:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572595.575430:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4512 00000100:00080000:1.0:1788572595.575452:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 10s 00000100:00080000:1.0:1788572595.575462:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572595.575479:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572595.575501:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ce00 x1875449616286464/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788572595.575580:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616286592:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572595.575677:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616286464:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572595.575749:0:2383:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1788572595.575763:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449616286336:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572595.581132:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2383:1875449616286336:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572595.583033:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616286592:192.168.202.113@tcp:400:kworker.0 00000100:00080000:2.0:1788572595.589629:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788572595.589650:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-114) 00000100:00100000:2.0:1788572595.589670:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616286464:192.168.202.113@tcp:38:kworker.0 00010000:00010000:1.0:1788572596.088233:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572597.111026:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572598.136154:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572599.159133:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572600.183051:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788572600.695532:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572600.695541:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788572600.695546:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572600.695559:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4513 00000100:00080000:1.0:1788572600.695573:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 15s 00000100:00100000:0.0:1788572600.695575:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449616286720:192.168.202.113@tcp:400:kworker.0 00000100:00080000:1.0:1788572600.695583:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572600.695596:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572600.695617:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ea00 x1875449616286848/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788572600.695677:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616286976:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572600.695693:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616286848:192.168.202.113@tcp:38:kworker.0 00010000:00010000:3.0:1788572601.207051:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572602.231091:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572603.254995:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572604.279040:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788572604.790910:0:92391:0:(recover.c:348:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery finished CONNECTING, rc = -110 00000080:00200008:1.0:1788572604.794879:0:92395:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1788572604.794892:0:92395:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:1.0:1788572604.794914:0:92395:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1788572604.794929:0:92395:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 36 lru_type: 1 00010000:00010000:1.0:1788572604.794944:0:92395:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 36 lru_type: 1 00000080:00200000:1.0:1788572604.794958:0:92395:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1788572604.794965:0:92395:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:1.0:1788572604.794969:0:92395:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:1.0:1788572604.794977:0:92395:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1c2fcb8 00010000:00010000:1.0:1788572604.794980:0:92395:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 36 lru_type: 1 00010000:00010000:1.0:1788572604.794987:0:92395:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 37 lru_type: 1 00000080:00200000:1.0:1788572604.795009:0:92395:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:1.0:1788572604.795015:0:92395:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1788572604.795023:0:92395:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 37 lru_type: 1 00010000:00010000:1.0:1788572604.795032:0:92395:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 37 lru_type: 1 00000080:00200000:1.0:1788572604.795038:0:92395:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1788572604.795042:0:92395:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:1.0:1788572604.795044:0:92395:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:1.0:1788572604.795048:0:92395:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1c2fbb0 00010000:00010000:1.0:1788572604.795052:0:92395:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 37 lru_type: 1 00010000:00010000:1.0:1788572604.795060:0:92395:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 38 lru_type: 1 00000080:00200000:1.0:1788572604.795068:0:92395:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:1.0:1788572604.795073:0:92395:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100400 00000080:00200008:1.0:1788572604.795083:0:92395:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200008:1.0:1788572604.795620:0:92395:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1788572604.795624:0:92395:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:1.0:1788572604.795630:0:92395:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1788572604.795635:0:92395:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 38 lru_type: 1 00010000:00010000:1.0:1788572604.795641:0:92395:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 38 lru_type: 1 00000080:00200000:1.0:1788572604.795645:0:92395:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1788572604.795648:0:92395:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:1.0:1788572604.795650:0:92395:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:1.0:1788572604.795653:0:92395:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1c2fcb8 00010000:00010000:1.0:1788572604.795655:0:92395:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 38 lru_type: 1 00010000:00010000:1.0:1788572604.795660:0:92395:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 39 lru_type: 1 00000080:00200008:1.0:1788572604.795674:0:92395:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1788572604.795676:0:92395:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:1.0:1788572604.795679:0:92395:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1788572604.795683:0:92395:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 39 lru_type: 1 00010000:00010000:1.0:1788572604.795688:0:92395:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 39 lru_type: 1 00000080:00200000:1.0:1788572604.795691:0:92395:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1788572604.795693:0:92395:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:1.0:1788572604.795695:0:92395:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:1.0:1788572604.795696:0:92395:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1c2fcb8 00010000:00010000:1.0:1788572604.795698:0:92395:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 39 lru_type: 1 00010000:00010000:1.0:1788572604.795705:0:92395:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 40 lru_type: 1 00000080:00200000:1.0:1788572604.795723:0:92395:0:(llite_lib.c:2903:ll_statfs()) VFS Op:sb=lustre (ffff9ada58b0a000) / proj=0 00000080:00000004:1.0:1788572604.795726:0:92395:0:(obd_class.h:1090:obd_statfs()) lustre-clilmv-ffff9ada58b0a000: age 4179, max_age 4521 00800000:00000004:1.0:1788572604.795736:0:92395:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: age 4488, max_age 4521 00000100:00100000:1.0:1788572604.795768:0:92395:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2840 going to sleep for 0 seconds 00000100:00100000:1.0:1788572604.795776:0:92395:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job df:35eb2d60-2de0-4b00-9815-d5153edd0057:92395:1875449616287104:192.168.202.113@tcp:41:df.0 00800000:00020000:1.0:1788572604.795788:0:92395:0:(lmv_obd.c:1471:lmv_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: can't stat MDS #0: rc = -114 00800000:00000004:1.0:1788572604.799049:0:92395:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: age 4488, max_age 4521 00000100:00100000:1.0:1788572604.799082:0:92395:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job df:35eb2d60-2de0-4b00-9815-d5153edd0057:92395:1875449616287232:192.168.202.113@tcp:41:df.0 00000100:00100000:1.0:1788572604.799145:0:92395:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2840 going to sleep for 12 seconds 00010000:00010000:1.0:1788572605.303010:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788572605.814990:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572605.814996:0:77466:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1788572605.815042:0:2384:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449616287360:192.168.202.113@tcp:400:kworker.0 00000100:00100000:1.0:1788572605.815042:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616287488:192.168.202.113@tcp:400:kworker.0 00010000:00010000:3.0:1788572606.327073:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572607.351024:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572608.375023:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572609.399001:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572610.423013:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788572610.935000:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572610.935006:0:85951:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1788572610.935039:0:2385:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449616287616:192.168.202.113@tcp:400:kworker.0 00000100:00100000:3.0:1788572610.935041:0:2386:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616287744:192.168.202.113@tcp:400:kworker.0 00010000:00010000:0.0:1788572611.447032:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572612.470999:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572613.494944:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572614.519013:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572615.542992:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000400:0.0:1788572616.054963:0:2384:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788572600/real 1788572600] req@ffff9ada4566ca80 x1875449616286720/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 1 dl 1788572616 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572616.054982:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572616.054988:0:91222:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1788572616.054994:0:2384:0:(client.c:2548:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9ada4566ca80 x1875449616286720/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 1 dl 1788572616 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572616.055002:0:2384:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from FULL to DISCONN 00000100:02020000:0.0:1788572616.055005:0:2384:0:(import.c:186:ptlrpc_set_import_discon()) MGC192.168.202.113@tcp: Connection to MGS (at 192.168.202.113@tcp) was lost; in progress operations using this service will fail 00000100:00080000:0.0:1788572616.055008:0:2384:0:(import.c:192:ptlrpc_set_import_discon()) import MGS@192.168.202.113@tcp for MGC192.168.202.113@tcp not replayable, auto-deactivating 00000100:00080000:0.0:1788572616.055009:0:2384:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import MGS INVALID 00000100:00100000:0.0:1788572616.055012:0:2384:0:(client.c:3591:ptlrpc_abort_inflight()) @@@ inflight req@ffff9ada4566ca80 x1875449616286720/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 1 dl 1788572616 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572616.055018:0:2384:0:(client.c:3591:ptlrpc_abort_inflight()) @@@ inflight req@ffff9ada4566f100 x1875449616287360/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572621 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572616.055024:0:2384:0:(client.c:3591:ptlrpc_abort_inflight()) @@@ inflight req@ffff9ada604e1f80 x1875449616287616/t0(0) o400->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 224/224 e 0 to 0 dl 1788572626 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572616.055031:0:2383:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616288000:192.168.202.113@tcp:400:kworker.0 10000000:01000000:0.0:1788572616.055038:0:2384:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808001 00000100:00080000:0.0:1788572616.055041:0:2384:0:(pinger.c:193:ptlrpc_pinger_ir_down()) IR down 10000000:01000000:0.0:1788572616.055041:0:2384:0:(mgc_request.c:1168:mgc_import_event()) import event 0x808002 00000100:00080000:0.0:1788572616.055043:0:2384:0:(import.c:418:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00100000:0.0:1788572616.055060:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:0:1875449616287872:192.168.202.113@tcp:400:kworker.0 00000100:00100000:0.0:1788572616.055071:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449616287360:192.168.202.113@tcp:400:kworker.0 00000100:00080000:3.0:1788572616.055082:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1788572616.055083:0:2384:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2384:1875449616286720:192.168.202.113@tcp:400:kworker.0 00000100:00100000:2.0:1788572616.055083:0:2385:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2385:1875449616287616:192.168.202.113@tcp:400:kworker.0 00000100:00080000:3.0:1788572616.055094:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788572616.055097:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572616.055105:0:91222:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4368 00000100:00080000:3.0:1788572616.055116:0:91222:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788572616.055136:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3100 x1875449616288128/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572616.055157:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572616.055161:0:91222:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1788572616.055192:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616288128:192.168.202.113@tcp:250:kworker.0 00010000:00010000:0.0:1788572616.566988:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572617.078966:0:92395:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2840 going to sleep for 1 seconds 00000100:00000400:0.0:1788572617.078979:0:2383:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788572600/real 1788572600] req@ffff9ada4566c380 x1875449616286976/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 1 dl 1788572617 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572617.079009:0:2383:0:(client.c:2548:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9ada4566c380 x1875449616286976/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 1 dl 1788572617 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572617.079019:0:2383:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from FULL to DISCONN 00000100:02000400:0.0:1788572617.079023:0:2383:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0001-mdc-ffff9ada58b0a000: Connection to lustre-MDT0001 (at 192.168.202.113@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:0.0:1788572617.079027:0:2383:0:(import.c:418:ptlrpc_pinger_force()) lustre-MDT0001_UUID: waking up pinger s:DISCONN 00000100:00100000:0.0:1788572617.079050:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616286976:192.168.202.113@tcp:400:kworker.0 00000100:00080000:3.0:1788572617.079108:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572617.079115:0:91222:0:(pinger.c:247:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1788572617.079118:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572617.079121:0:91222:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1788572617.079123:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572617.079127:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788572617.079130:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572617.079135:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 706 00000100:00080000:3.0:1788572617.079145:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572617.079154:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572617.079175:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0a80 x1875449616288256/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788572617.079235:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616288256:192.168.202.113@tcp:38:kworker.0 00010000:00010000:0.0:1788572617.591046:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788572618.103001:0:92395:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2840 going to sleep for 1 seconds 00010000:00010000:0.0:1788572618.614987:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788572619.126938:0:92395:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2840 going to sleep for 1 seconds 00010000:00010000:0.0:1788572619.639017:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788572620.150960:0:92395:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2840 going to sleep for 1 seconds 00010000:00010000:0.0:1788572620.662973:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788572621.111301:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572621.111311:0:91222:0:(pinger.c:247:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1788572621.111315:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572621.111318:0:91222:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1788572621.111321:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572621.111324:0:91222:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00000400:2.0:1788572621.174984:0:92395:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788572604/real 1788572604] req@ffff9ada4566d500 x1875449616287232/t0(0) o41->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 440/368 e 0 to 1 dl 1788572621 ref 2 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'df.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788572621.175023:0:92395:0:(client.c:2548:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9ada4566d500 x1875449616287232/t0(0) o41->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 440/368 e 0 to 1 dl 1788572621 ref 2 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'df.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788572621.175033:0:92395:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff9ada58b08000 already not connected (conn 2, was 1): CONNECTING 00000100:00100000:2.0:1788572621.175040:0:92395:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2840 going to sleep for 0 seconds 00000100:00100000:2.0:1788572621.175048:0:92395:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job df:35eb2d60-2de0-4b00-9815-d5153edd0057:92395:1875449616287232:192.168.202.113@tcp:41:df.0 00800000:00020000:2.0:1788572621.175075:0:92395:0:(lmv_obd.c:1471:lmv_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: can't stat MDS #1: rc = -11 00010000:00010000:0.0:1788572621.687066:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788572622.134963:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572622.134971:0:91222:0:(pinger.c:247:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1788572622.134973:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572622.134976:0:91222:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1788572622.134978:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572622.134980:0:91222:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00000400:0.0:1788572622.198924:0:2383:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788572605/real 1788572605] req@ffff9ada4566d180 x1875449616287488/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 1 dl 1788572622 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572622.198960:0:2383:0:(client.c:2548:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9ada4566d180 x1875449616287488/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 1 dl 1788572622 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572622.198968:0:2383:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff9ada58b08000 already not connected (conn 2, was 1): CONNECTING 00000100:00100000:0.0:1788572622.198980:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616287488:192.168.202.113@tcp:400:kworker.0 00010000:00010000:0.0:1788572622.711068:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572623.735043:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572624.759014:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572625.270950:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788572600/real 1788572600] req@ffff9ada4566ea00 x1875449616286848/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572625 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572625.270985:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ea00 x1875449616286848/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572625 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572625.271001:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572625.271024:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572625.271028:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616286848:192.168.202.113@tcp:38:kworker.0 00000100:00080000:3.0:1788572625.271076:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572625.271083:0:91222:0:(pinger.c:247:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1788572625.271086:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572625.271091:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788572625.271095:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572625.271102:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4518 00000100:00080000:3.0:1788572625.271111:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572625.271119:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572625.271130:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572625.271154:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1880 x1875449616288384/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572625.271176:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572625.271179:0:91222:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1788572625.271230:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616288384:192.168.202.113@tcp:38:kworker.0 00010000:00010000:0.0:1788572625.783058:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572626.294976:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788572616/real 1788572616] req@ffff9ada604e3100 x1875449616288128/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572626 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572626.295009:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3100 x1875449616288128/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572626 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572626.295043:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572626.295066:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572626.295071:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616288128:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788572626.295107:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572626.295113:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788572626.295115:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572626.295121:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4533 00000100:00080000:3.0:1788572626.295129:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 5s 00000100:00080000:3.0:1788572626.295135:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788572626.295167:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0000 x1875449616288512/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572626.295190:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572626.295193:0:85951:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1788572626.295195:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572626.295198:0:85951:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1788572626.295234:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616288512:192.168.202.113@tcp:250:kworker.0 00010000:00010000:0.0:1788572626.807076:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572627.318981:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788572617/real 1788572617] req@ffff9ada604e0a80 x1875449616288256/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572627 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000400:2.0:1788572627.318987:0:2386:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788572610/real 1788572610] req@ffff9ada604e1c00 x1875449616287744/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 1 dl 1788572627 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788572627.319043:0:2386:0:(client.c:2548:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9ada604e1c00 x1875449616287744/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 1 dl 1788572627 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788572627.319052:0:2386:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff9ada58b08000 already not connected (conn 2, was 1): CONNECTING 00000100:00100000:1.0:1788572627.319053:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0a80 x1875449616288256/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572627 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572627.319079:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:2.0:1788572627.319080:0:2386:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:35eb2d60-2de0-4b00-9815-d5153edd0057:2386:1875449616287744:192.168.202.113@tcp:400:kworker.0 00000100:00080000:1.0:1788572627.319110:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572627.319116:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616288256:192.168.202.113@tcp:38:kworker.0 00000100:00080000:3.0:1788572627.319241:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572627.319249:0:85951:0:(pinger.c:247:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1788572627.319253:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572627.319257:0:85951:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1788572627.319262:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572627.319267:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788572627.319270:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572627.319278:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4534 00000100:00080000:3.0:1788572627.319290:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 5s 00000100:00080000:3.0:1788572627.319299:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572627.319315:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572627.319350:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2d80 x1875449616288640/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788572627.319403:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616288640:192.168.202.113@tcp:38:kworker.0 00010000:00010000:0.0:1788572627.831024:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572628.855024:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572629.879059:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788572630.326989:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572630.327001:0:85951:0:(pinger.c:247:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1788572630.327004:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572630.327012:0:85951:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1788572630.327018:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572630.327021:0:85951:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:0.0:1788572630.903026:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788572631.350955:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572631.350964:0:85951:0:(pinger.c:247:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1788572631.350967:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572631.350970:0:85951:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1788572631.350972:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572631.350975:0:85951:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:0.0:1788572631.927059:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000800:00000100:1.0:1788572632.566937:0:2374:0:(socklnd_cb.c:2254:ksocknal_find_timed_out_conn()) A connection with 12345-192.168.202.113@tcp (192.168.202.113:988) timed out; the network or node may be down. 00010000:00010000:0.0:1788572632.951082:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000400:0.0:1788572633.462963:0:2383:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788572616/real 1788572616] req@ffff9ada4566f480 x1875449616288000/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 1 dl 1788572633 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572633.462992:0:2383:0:(client.c:2548:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff9ada4566f480 x1875449616288000/t0(0) o400->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/224 e 0 to 1 dl 1788572633 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788572633.463000:0:2383:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff9ada58b08000 already not connected (conn 3, was 1): CONNECTING 00000100:00100000:0.0:1788572633.463015:0:2383:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:35eb2d60-2de0-4b00-9815-d5153edd0057:2383:1875449616288000:192.168.202.113@tcp:400:kworker.0 00010000:00010000:0.0:1788572633.975002:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572634.999026:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572636.023036:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788572636.534986:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572636.534994:0:85951:0:(pinger.c:247:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1788572636.534997:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572636.534999:0:85951:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1788572636.535001:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572636.535003:0:85951:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:0.0:1788572637.047049:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572638.071029:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:2.0:1788572638.978358:0:92418:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=40086685 len=0 uarg=00007fff16fb5060 00000002:00000080:2.0:1788572638.978383:0:92418:0:(mdc_request.c:2317:mdc_iocontrol()) lustre-MDT0000-mdc-ffff9ada58b0a000: cmd=40086685 len=576 karg=ffff9ada4566a800 uarg=0000000000000000 00000100:00080000:2.0:1788572638.978393:0:92418:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff9ada51140000 already not connected (conn 364, was 0): CONNECTING 00000100:00080000:2.0:1788572638.978400:0:92418:0:(recover.c:337:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery started, waiting 20 seconds 00010000:00010000:0.0:1788572639.095068:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572640.119026:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572641.143053:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572641.590926:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788572626/real 1788572626] req@ffff9ada604e0000 x1875449616288512/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572641 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572641.590952:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0000 x1875449616288512/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572641 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572641.590969:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572641.590988:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572641.590991:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616288512:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788572641.591044:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572641.591052:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788572641.591055:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572641.591062:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4543 00000100:00080000:2.0:1788572641.591073:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 10s 00000100:00080000:2.0:1788572641.591079:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788572641.591109:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616288768/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788572641.591129:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level CONNECTING/4 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572641.591133:0:85951:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1788572641.591135:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572641.591138:0:85951:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1788572641.591157:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616288768:192.168.202.113@tcp:250:kworker.0 00000400:00000100:2.0:1788572642.039203:0:2370:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1022 -> 192.168.202.113:988 00000400:00000100:2.0:1788572642.039269:0:2370:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1788572642.039290:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:1.0:1788572642.039386:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572641/real 1788572642] req@ffff9ada60627480 x1875449616288768/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572661 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572642.039426:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616288768/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572661 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572642.039441:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572642.039447:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572642.039452:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616288768:192.168.202.113@tcp:250:kworker.0 00010000:00010000:0.0:1788572642.167022:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788572642.614948:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788572627/real 1788572627] req@ffff9ada604e2d80 x1875449616288640/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572642 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572642.614974:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2d80 x1875449616288640/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572642 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572642.614989:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572642.615014:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572642.615017:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616288640:192.168.202.113@tcp:38:kworker.0 00000100:00080000:3.0:1788572642.615051:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572642.615057:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788572642.615060:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572642.615065:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4559 00000100:00080000:3.0:1788572642.615074:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 15s 00000100:00080000:3.0:1788572642.615080:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788572642.615111:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2300 x1875449616288896/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572642.615137:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level CONNECTING/4 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572642.615142:0:85951:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1788572642.615145:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572642.615149:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788572642.615151:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572642.615157:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4544 00000100:00080000:3.0:1788572642.615162:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 10s 00000100:00100000:1.0:1788572642.615166:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616288896:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788572642.615167:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572642.615180:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572642.615194:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3b80 x1875449616289024/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788572642.615252:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616289024:192.168.202.113@tcp:38:kworker.0 00010000:00010000:0.0:1788572643.191065:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572644.215049:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1788572645.111300:0:2372:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1022 -> 192.168.202.113:988 00000400:00000100:0.0:1788572645.111371:0:2372:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1788572645.111388:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:0.0:1788572645.111437:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:1.0:1788572645.111516:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572642/real 1788572645] req@ffff9ada604e3b80 x1875449616289024/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572662 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572645.111542:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3b80 x1875449616289024/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572662 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572645.111554:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572645.111559:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572645.111564:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616289024:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572645.111578:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572642/real 1788572645] req@ffff9ada604e2300 x1875449616288896/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572667 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572645.111592:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2300 x1875449616288896/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572667 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572645.111601:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572645.111605:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572645.111608:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616288896:192.168.202.113@tcp:250:kworker.0 00010000:00010000:0.0:1788572645.239043:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572646.263006:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572647.287168:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788572647.799011:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572647.799021:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788572647.799025:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572647.799035:0:91222:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4560 00000100:00080000:1.0:1788572647.799045:0:91222:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572647.799053:0:91222:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788572647.799097:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ea00 x1875449616289152/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572647.799125:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level CONNECTING/4 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572647.799130:0:91222:0:(pinger.c:247:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1788572647.799134:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572647.799138:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788572647.799140:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572647.799145:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4560 00000100:00080000:1.0:1788572647.799150:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 15s 00000100:00080000:1.0:1788572647.799155:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572647.799164:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572647.799173:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616289152:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788572647.799179:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e300 x1875449616289280/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788572647.799271:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616289280:192.168.202.113@tcp:38:kworker.0 00000400:00000100:2.0:1788572648.183205:0:2373:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1022 -> 192.168.202.113:988 00000400:00000100:2.0:1788572648.183253:0:2373:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1788572648.183271:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788572648.183318:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:3.0:1788572648.183410:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572647/real 1788572648] req@ffff9ada4566e300 x1875449616289280/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572672 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572648.183437:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e300 x1875449616289280/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572672 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572648.183451:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572648.183456:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572648.183460:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616289280:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572648.183469:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572647/real 1788572648] req@ffff9ada4566ea00 x1875449616289152/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572672 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572648.183483:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ea00 x1875449616289152/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572672 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572648.183492:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572648.183495:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572648.183499:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616289152:192.168.202.113@tcp:250:kworker.0 00010000:00010000:0.0:1788572648.311001:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572649.335005:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572650.359046:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788572650.870999:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788572625/real 1788572625] req@ffff9ada604e1880 x1875449616288384/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572650 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572650.871055:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1880 x1875449616288384/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572650 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572650.871080:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572650.871109:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572650.871127:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616288384:192.168.202.113@tcp:38:kworker.0 00000100:00080000:1.0:1788572650.871193:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572650.871203:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788572650.871208:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572650.871218:0:91222:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4565 00000100:00080000:1.0:1788572650.871229:0:91222:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572650.871237:0:91222:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788572650.871290:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ed80 x1875449616289408/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572650.871314:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572650.871317:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788572650.871319:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572650.871322:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4542 00000100:00080000:1.0:1788572650.871324:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572650.871328:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572650.871336:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572650.871347:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f800 x1875449616289536/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572650.871356:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1788572650.871363:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616289408:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788572650.871364:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788572650.871367:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572650.871371:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4565 00000100:00080000:1.0:1788572650.871373:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572650.871375:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572650.871382:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572650.871400:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566dc00 x1875449616289664/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788572650.871471:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572650.871509:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572650/real 1788572650] req@ffff9ada4566ed80 x1875449616289408/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572675 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572650.871527:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ed80 x1875449616289408/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572675 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572650.871538:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572650.871542:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572650.871546:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616289408:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572650.871562:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616289536:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572650.871587:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572650.871600:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572650/real 1788572650] req@ffff9ada4566f800 x1875449616289536/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572675 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572650.871614:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f800 x1875449616289536/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572675 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572650.871642:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572650.871646:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572650.871660:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616289536:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572650.871668:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616289664:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572650.871695:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572650.871710:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572650/real 1788572650] req@ffff9ada4566dc00 x1875449616289664/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572675 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572650.871724:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566dc00 x1875449616289664/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572675 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572650.871733:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572650.871737:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572650.871740:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616289664:192.168.202.113@tcp:38:kworker.0 00010000:00010000:0.0:1788572651.383022:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572652.407044:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572653.431029:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572654.455125:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572655.479013:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788572655.990999:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572655.991010:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788572655.991013:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572655.991031:0:91222:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4568 00000100:00080000:1.0:1788572655.991042:0:91222:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572655.991049:0:91222:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788572655.991099:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c000 x1875449616289792/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572655.991138:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572655.991142:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788572655.991143:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572655.991146:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4568 00000100:00080000:1.0:1788572655.991150:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572655.991152:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572655.991162:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572655.991175:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566df80 x1875449616289920/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572655.991186:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572655.991190:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788572655.991191:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572655.991195:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4568 00000100:00080000:1.0:1788572655.991199:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572655.991203:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572655.991216:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572655.991228:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566fb80 x1875449616290048/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788572655.991231:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616289792:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572655.991343:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616289920:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572655.991383:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616290048:192.168.202.113@tcp:38:kworker.0 00010000:00010000:1.0:1788572656.503063:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1788572657.399233:0:2371:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1022 -> 192.168.202.113:988 00000400:00000100:0.0:1788572657.399289:0:2371:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1788572657.399310:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:0.0:1788572657.399354:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:0.0:1788572657.399366:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:3.0:1788572657.399399:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572655/real 1788572657] req@ffff9ada4566fb80 x1875449616290048/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572680 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572657.399432:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566fb80 x1875449616290048/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572680 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572657.399451:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572657.399457:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572657.399461:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616290048:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572657.399469:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572655/real 1788572657] req@ffff9ada4566c000 x1875449616289792/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572680 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572657.399481:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c000 x1875449616289792/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572680 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572657.399488:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572657.399492:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572657.399494:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616289792:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572657.399502:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572655/real 1788572657] req@ffff9ada4566df80 x1875449616289920/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572680 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572657.399531:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566df80 x1875449616289920/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572680 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572657.399538:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572657.399542:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572657.399564:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616289920:192.168.202.113@tcp:38:kworker.0 00010000:00010000:3.0:1788572657.527031:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572658.551028:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788572658.998951:0:92418:0:(recover.c:348:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery finished DISCONN, rc = -110 00000080:00200008:2.0:1788572659.002990:0:92422:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1788572659.003006:0:92422:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788572659.003045:0:92422:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788572659.003068:0:92422:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 40 lru_type: 1 00010000:00010000:2.0:1788572659.003084:0:92422:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 40 lru_type: 1 00000080:00200000:2.0:1788572659.003094:0:92422:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788572659.003103:0:92422:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788572659.003106:0:92422:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:2.0:1788572659.003115:0:92422:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1cafcb8 00010000:00010000:2.0:1788572659.003120:0:92422:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 40 lru_type: 1 00010000:00010000:2.0:1788572659.003130:0:92422:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 41 lru_type: 1 00000080:00200000:2.0:1788572659.003158:0:92422:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788572659.003163:0:92422:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788572659.003167:0:92422:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 41 lru_type: 1 00010000:00010000:2.0:1788572659.003173:0:92422:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 41 lru_type: 1 00000080:00200000:2.0:1788572659.003177:0:92422:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788572659.003180:0:92422:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788572659.003182:0:92422:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:2.0:1788572659.003184:0:92422:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1cafbb0 00010000:00010000:2.0:1788572659.003186:0:92422:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 41 lru_type: 1 00010000:00010000:2.0:1788572659.003191:0:92422:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 42 lru_type: 1 00000080:00200000:2.0:1788572659.003195:0:92422:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:2.0:1788572659.003200:0:92422:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100400 00000080:00200008:2.0:1788572659.003211:0:92422:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200008:2.0:1788572659.003820:0:92422:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1788572659.003823:0:92422:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788572659.003829:0:92422:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788572659.003835:0:92422:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 42 lru_type: 1 00010000:00010000:2.0:1788572659.003841:0:92422:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 42 lru_type: 1 00000080:00200000:2.0:1788572659.003845:0:92422:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788572659.003848:0:92422:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788572659.003849:0:92422:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:2.0:1788572659.003885:0:92422:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1cafcb8 00010000:00010000:2.0:1788572659.003889:0:92422:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 42 lru_type: 1 00010000:00010000:2.0:1788572659.003904:0:92422:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 43 lru_type: 1 00000080:00200008:2.0:1788572659.003930:0:92422:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1788572659.003933:0:92422:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788572659.003939:0:92422:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788572659.003944:0:92422:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 43 lru_type: 1 00010000:00010000:2.0:1788572659.003950:0:92422:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 43 lru_type: 1 00000080:00200000:2.0:1788572659.003954:0:92422:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788572659.003957:0:92422:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788572659.003958:0:92422:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:2.0:1788572659.003961:0:92422:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1cafcb8 00010000:00010000:2.0:1788572659.003963:0:92422:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 43 lru_type: 1 00010000:00010000:2.0:1788572659.003967:0:92422:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 44 lru_type: 1 00000080:00200000:2.0:1788572659.003981:0:92422:0:(llite_lib.c:2903:ll_statfs()) VFS Op:sb=lustre (ffff9ada58b0a000) / proj=0 00000080:00000004:2.0:1788572659.003986:0:92422:0:(obd_class.h:1090:obd_statfs()) lustre-clilmv-ffff9ada58b0a000: age 4179, max_age 4575 00800000:00000004:2.0:1788572659.003995:0:92422:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: age 4488, max_age 4575 00000100:00100000:2.0:1788572659.004049:0:92422:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60703240 going to sleep for 0 seconds 00000100:00100000:2.0:1788572659.004061:0:92422:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job df:35eb2d60-2de0-4b00-9815-d5153edd0057:92422:1875449616290176:192.168.202.113@tcp:41:df.0 00800000:00020000:2.0:1788572659.004075:0:92422:0:(lmv_obd.c:1471:lmv_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: can't stat MDS #0: rc = -110 00800000:00000004:2.0:1788572659.006835:0:92422:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: age 4488, max_age 4575 00000100:00100000:2.0:1788572659.006890:0:92422:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60703240 going to sleep for 0 seconds 00000100:00100000:2.0:1788572659.006900:0:92422:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job df:35eb2d60-2de0-4b00-9815-d5153edd0057:92422:1875449616290304:192.168.202.113@tcp:41:df.0 00800000:00020000:2.0:1788572659.006912:0:92422:0:(lmv_obd.c:1471:lmv_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: can't stat MDS #1: rc = -110 00010000:00010000:1.0:1788572659.575003:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572660.599025:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788572661.110973:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572661.110985:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788572661.110991:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572661.111009:0:91222:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4573 00000100:00080000:1.0:1788572661.111022:0:91222:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572661.111031:0:91222:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788572661.111077:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566df80 x1875449616290432/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572661.111114:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572661.111118:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788572661.111120:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572661.111124:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4573 00000100:00080000:1.0:1788572661.111128:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572661.111133:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572661.111145:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572661.111158:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616290432:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788572661.111170:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c000 x1875449616290560/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572661.111183:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572661.111186:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788572661.111188:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572661.111192:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4573 00000100:00080000:1.0:1788572661.111196:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572661.111200:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572661.111205:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572661.111215:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566fb80 x1875449616290688/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788572661.111241:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572661.111285:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572661/real 1788572661] req@ffff9ada4566df80 x1875449616290432/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572686 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572661.111302:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566df80 x1875449616290432/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572686 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572661.111312:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572661.111316:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572661.111321:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616290432:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572661.111349:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616290560:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572661.111370:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572661.111380:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572661/real 1788572661] req@ffff9ada4566c000 x1875449616290560/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572686 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572661.111388:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c000 x1875449616290560/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572686 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572661.111393:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572661.111395:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572661.111397:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616290560:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572661.111400:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616290688:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572661.111422:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572661.111433:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572661/real 1788572661] req@ffff9ada4566fb80 x1875449616290688/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572686 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572661.111445:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566fb80 x1875449616290688/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572686 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572661.111457:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572661.111461:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572661.111464:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616290688:192.168.202.113@tcp:38:kworker.0 00010000:00010000:3.0:1788572661.623013:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572662.647001:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:1.0:1788572663.551527:0:92441:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=40086685 len=0 uarg=00007ffcaf9fea40 00000002:00000080:1.0:1788572663.551543:0:92441:0:(mdc_request.c:2317:mdc_iocontrol()) lustre-MDT0000-mdc-ffff9ada58b0a000: cmd=40086685 len=576 karg=ffff9ada5164b400 uarg=0000000000000000 00000100:00080000:1.0:1788572663.551549:0:92441:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff9ada51140000 already not connected (conn 367, was 0): DISCONN 00000100:00080000:1.0:1788572663.551555:0:92441:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572663.551562:0:92441:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4578 00000100:00080000:1.0:1788572663.551569:0:92441:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572663.551574:0:92441:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572663.551582:0:92441:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572663.551602:0:92441:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566dc00 x1875449616290816/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572663.551623:0:92441:0:(recover.c:337:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery started, waiting 20 seconds 00000100:00100000:3.0:1788572663.551657:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616290816:192.168.202.113@tcp:38:lctl.0 00000800:00000100:3.0:1788572663.551710:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572663.551733:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572663/real 1788572663] req@ffff9ada4566dc00 x1875449616290816/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572688 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'lctl.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572663.551755:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566dc00 x1875449616290816/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572688 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'lctl.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572663.551765:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572663.551769:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572663.551782:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616290816:192.168.202.113@tcp:38:lctl.0 00010000:00010000:3.0:1788572663.670978:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572664.695054:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572665.718984:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788572666.230960:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572666.230968:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788572666.230972:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572666.230980:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4578 00000100:00080000:3.0:1788572666.230989:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572666.230995:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788572666.231027:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1880 x1875449616290944/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572666.231051:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572666.231054:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788572666.231055:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572666.231058:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4581 00000100:00080000:3.0:1788572666.231061:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572666.231064:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572666.231072:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572666.231083:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616290944:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572666.231084:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3800 x1875449616291072/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572666.231095:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572666.231099:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788572666.231101:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572666.231103:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4578 00000100:00080000:3.0:1788572666.231105:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572666.231108:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572666.231110:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572666.231118:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1180 x1875449616291200/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788572666.231181:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616291072:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572666.231202:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616291200:192.168.202.113@tcp:38:kworker.0 00000400:00000100:2.0:1788572666.615210:0:2370:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1022 -> 192.168.202.113:988 00000400:00000100:2.0:1788572666.615272:0:2370:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1788572666.615291:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788572666.615341:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788572666.615354:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:1.0:1788572666.615424:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572666/real 1788572666] req@ffff9ada604e3800 x1875449616291072/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572691 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572666.615463:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3800 x1875449616291072/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572691 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572666.615477:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572666.615483:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572666.615508:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616291072:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572666.615515:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572666/real 1788572666] req@ffff9ada604e1180 x1875449616291200/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572691 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572666.615526:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1180 x1875449616291200/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572691 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572666.615532:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572666.615535:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572666.615538:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616291200:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572666.615543:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572666/real 1788572666] req@ffff9ada604e1880 x1875449616290944/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572691 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572666.615550:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1880 x1875449616290944/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572691 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572666.615555:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572666.615557:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572666.615559:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616290944:192.168.202.113@tcp:250:kworker.0 00010000:00010000:3.0:1788572666.743062:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572667.767007:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572668.790990:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572669.815018:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572670.838982:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788572671.350956:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572671.350964:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788572671.350968:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572671.350977:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4583 00000100:00080000:3.0:1788572671.350986:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572671.350992:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788572671.351025:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0380 x1875449616291328/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572671.351050:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572671.351053:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788572671.351054:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572671.351057:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4583 00000100:00080000:3.0:1788572671.351059:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572671.351062:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572671.351069:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572671.351078:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0700 x1875449616291456/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572671.351088:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572671.351092:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788572671.351094:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572671.351102:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4583 00000100:00100000:1.0:1788572671.351103:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616291328:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788572671.351108:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572671.351114:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572671.351129:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572671.351145:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3480 x1875449616291584/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788572671.351161:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572671.351194:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572671/real 1788572671] req@ffff9ada604e0380 x1875449616291328/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572696 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572671.351209:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0380 x1875449616291328/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572696 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572671.351220:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572671.351224:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572671.351228:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616291328:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788572671.351235:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616291456:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572671.351270:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572671.351285:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572671/real 1788572671] req@ffff9ada604e0700 x1875449616291456/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572696 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572671.351305:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0700 x1875449616291456/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572696 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572671.351312:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572671.351314:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572671.351324:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616291456:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572671.351344:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616291584:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572671.351364:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572671.351373:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572671/real 1788572671] req@ffff9ada604e3480 x1875449616291584/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572696 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572671.351380:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3480 x1875449616291584/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572696 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572671.351386:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572671.351388:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572671.351390:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616291584:192.168.202.113@tcp:38:kworker.0 00010000:00010000:0.0:1788572671.863017:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572672.887007:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572673.911021:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572674.935004:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572675.959001:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788572676.470913:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572676.470921:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788572676.470924:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572676.470933:0:91222:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4589 00000100:00080000:3.0:1788572676.470942:0:91222:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572676.470948:0:91222:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788572676.470981:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3480 x1875449616291712/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572676.471010:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572676.471014:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788572676.471016:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572676.471020:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4589 00000100:00080000:3.0:1788572676.471024:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572676.471029:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572676.471040:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572676.471056:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616291712:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572676.471063:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0700 x1875449616291840/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572676.471073:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572676.471075:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788572676.471076:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572676.471078:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4589 00000100:00080000:3.0:1788572676.471081:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572676.471084:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572676.471089:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572676.471098:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0380 x1875449616291968/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788572676.471121:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572676.471155:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572676/real 1788572676] req@ffff9ada604e3480 x1875449616291712/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572701 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572676.471191:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3480 x1875449616291712/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572701 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572676.471202:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572676.471207:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572676.471211:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616291712:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788572676.471233:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616291840:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572676.471253:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572676.471264:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572676/real 1788572676] req@ffff9ada604e0700 x1875449616291840/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572701 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572676.471277:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0700 x1875449616291840/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572701 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572676.471286:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572676.471289:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572676.471302:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616291840:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572676.471308:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616291968:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572676.471333:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572676.471346:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572676/real 1788572676] req@ffff9ada604e0380 x1875449616291968/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572701 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572676.471358:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0380 x1875449616291968/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572701 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572676.471373:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572676.471376:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572676.471379:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616291968:192.168.202.113@tcp:38:kworker.0 00010000:00010000:0.0:1788572676.983030:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572678.007059:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572679.031010:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572680.055036:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572681.079038:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788572681.590943:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572681.590951:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788572681.590955:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572681.590963:0:91222:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4594 00000100:00080000:3.0:1788572681.590972:0:91222:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572681.590978:0:91222:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788572681.591008:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1880 x1875449616292096/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572681.591041:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572681.591045:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788572681.591052:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572681.591056:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4594 00000100:00080000:3.0:1788572681.591060:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572681.591065:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572681.591075:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572681.591088:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1180 x1875449616292224/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572681.591097:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572681.591099:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788572681.591101:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572681.591104:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4594 00000100:00080000:3.0:1788572681.591108:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572681.591111:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788572681.591116:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616292096:192.168.202.113@tcp:250:kworker.0 00000008:00100000:3.0:1788572681.591126:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572681.591139:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3800 x1875449616292352/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788572681.591173:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572681.591203:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572681/real 1788572681] req@ffff9ada604e1880 x1875449616292096/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572706 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572681.591217:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1880 x1875449616292096/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572706 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572681.591225:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572681.591227:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572681.591230:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616292096:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788572681.591236:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616292224:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572681.591250:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572681.591257:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572681/real 1788572681] req@ffff9ada604e1180 x1875449616292224/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572706 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572681.591265:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1180 x1875449616292224/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572706 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572681.591271:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572681.591273:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572681.591286:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616292224:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572681.591307:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616292352:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572681.591324:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572681.591333:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572681/real 1788572681] req@ffff9ada604e3800 x1875449616292352/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572706 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572681.591358:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3800 x1875449616292352/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572706 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572681.591363:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572681.591366:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572681.591368:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616292352:192.168.202.113@tcp:38:kworker.0 00010000:00010000:0.0:1788572682.103129:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572683.127015:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788572683.574937:0:92441:0:(recover.c:348:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery finished DISCONN, rc = -110 00000080:00200008:3.0:1788572683.578848:0:92445:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1788572683.578874:0:92445:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:3.0:1788572683.578893:0:92445:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1788572683.578910:0:92445:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 44 lru_type: 1 00010000:00010000:3.0:1788572683.578923:0:92445:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 44 lru_type: 1 00000080:00200000:3.0:1788572683.578930:0:92445:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1788572683.578936:0:92445:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:3.0:1788572683.578939:0:92445:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:3.0:1788572683.578945:0:92445:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1cc7cb8 00010000:00010000:3.0:1788572683.578948:0:92445:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 44 lru_type: 1 00010000:00010000:3.0:1788572683.578955:0:92445:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 45 lru_type: 1 00000080:00200000:3.0:1788572683.578976:0:92445:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:3.0:1788572683.578980:0:92445:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1788572683.578985:0:92445:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 45 lru_type: 1 00010000:00010000:3.0:1788572683.578990:0:92445:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 45 lru_type: 1 00000080:00200000:3.0:1788572683.578995:0:92445:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1788572683.578998:0:92445:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:3.0:1788572683.578999:0:92445:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:3.0:1788572683.579001:0:92445:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1cc7bb0 00010000:00010000:3.0:1788572683.579003:0:92445:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 45 lru_type: 1 00010000:00010000:3.0:1788572683.579008:0:92445:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 46 lru_type: 1 00000080:00200000:3.0:1788572683.579014:0:92445:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:3.0:1788572683.579019:0:92445:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100400 00000080:00200008:3.0:1788572683.579029:0:92445:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200008:3.0:1788572683.579592:0:92445:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1788572683.579598:0:92445:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:3.0:1788572683.579606:0:92445:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1788572683.579614:0:92445:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 46 lru_type: 1 00010000:00010000:3.0:1788572683.579624:0:92445:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 46 lru_type: 1 00000080:00200000:3.0:1788572683.579639:0:92445:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1788572683.579643:0:92445:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:3.0:1788572683.579646:0:92445:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:3.0:1788572683.579651:0:92445:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1cc7cb8 00010000:00010000:3.0:1788572683.579653:0:92445:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 46 lru_type: 1 00010000:00010000:3.0:1788572683.579659:0:92445:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 47 lru_type: 1 00000080:00200008:3.0:1788572683.579678:0:92445:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1788572683.579680:0:92445:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:3.0:1788572683.579685:0:92445:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1788572683.579689:0:92445:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 47 lru_type: 1 00010000:00010000:3.0:1788572683.579695:0:92445:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 47 lru_type: 1 00000080:00200000:3.0:1788572683.579699:0:92445:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1788572683.579701:0:92445:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:3.0:1788572683.579703:0:92445:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:3.0:1788572683.579705:0:92445:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1cc7cb8 00010000:00010000:3.0:1788572683.579707:0:92445:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 47 lru_type: 1 00010000:00010000:3.0:1788572683.579716:0:92445:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 48 lru_type: 1 00000080:00200000:3.0:1788572683.579729:0:92445:0:(llite_lib.c:2903:ll_statfs()) VFS Op:sb=lustre (ffff9ada58b0a000) / proj=0 00000080:00000004:3.0:1788572683.579732:0:92445:0:(obd_class.h:1090:obd_statfs()) lustre-clilmv-ffff9ada58b0a000: age 4179, max_age 4600 00800000:00000004:3.0:1788572683.579739:0:92445:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: age 4488, max_age 4600 00000100:00100000:3.0:1788572683.579770:0:92445:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6059b0c0 going to sleep for 0 seconds 00000100:00100000:3.0:1788572683.579778:0:92445:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job df:35eb2d60-2de0-4b00-9815-d5153edd0057:92445:1875449616292480:192.168.202.113@tcp:41:df.0 00800000:00020000:3.0:1788572683.579791:0:92445:0:(lmv_obd.c:1471:lmv_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: can't stat MDS #0: rc = -110 00800000:00000004:3.0:1788572683.586397:0:92445:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: age 4488, max_age 4600 00000100:00100000:3.0:1788572683.586514:0:92445:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6059b0c0 going to sleep for 0 seconds 00000100:00100000:3.0:1788572683.586526:0:92445:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job df:35eb2d60-2de0-4b00-9815-d5153edd0057:92445:1875449616292608:192.168.202.113@tcp:41:df.0 00800000:00020000:3.0:1788572683.586539:0:92445:0:(lmv_obd.c:1471:lmv_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: can't stat MDS #1: rc = -110 00010000:00010000:0.0:1788572684.151025:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572685.175045:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572686.199099:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788572686.710979:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572686.710989:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788572686.710993:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572686.711003:0:91222:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4599 00000100:00080000:3.0:1788572686.711013:0:91222:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572686.711020:0:91222:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788572686.711059:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2300 x1875449616292736/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572686.711092:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572686.711096:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788572686.711098:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572686.711103:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4599 00000100:00080000:3.0:1788572686.711107:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572686.711111:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572686.711121:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572686.711136:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3b80 x1875449616292864/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788572686.711146:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616292736:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788572686.711152:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572686.711158:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788572686.711160:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572686.711165:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4599 00000100:00080000:3.0:1788572686.711169:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572686.711174:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572686.711180:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572686.711193:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2d80 x1875449616292992/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788572686.711254:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616292864:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572686.711280:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616292992:192.168.202.113@tcp:38:kworker.0 00010000:00010000:0.0:1788572687.223063:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1788572688.119205:0:2372:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1022 -> 192.168.202.113:988 00000400:00000100:0.0:1788572688.119245:0:2372:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1788572688.119263:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:0.0:1788572688.119305:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:0.0:1788572688.119318:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:3.0:1788572688.119347:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572686/real 1788572688] req@ffff9ada604e3b80 x1875449616292864/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572711 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572688.119376:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3b80 x1875449616292864/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572711 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572688.119392:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572688.119397:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572688.119402:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616292864:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572688.119413:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572686/real 1788572688] req@ffff9ada604e2d80 x1875449616292992/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572711 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572688.119425:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2d80 x1875449616292992/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572711 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572688.119437:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572688.119440:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572688.119444:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616292992:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572688.119451:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572686/real 1788572688] req@ffff9ada604e2300 x1875449616292736/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572711 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572688.119464:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2300 x1875449616292736/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572711 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572688.119472:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572688.119475:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572688.119478:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616292736:192.168.202.113@tcp:250:kworker.0 00000020:00000080:3.0:1788572688.128992:0:92463:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=40086685 len=0 uarg=00007ffde3c2e9f0 00000002:00000080:3.0:1788572688.129009:0:92463:0:(mdc_request.c:2317:mdc_iocontrol()) lustre-MDT0000-mdc-ffff9ada58b0a000: cmd=40086685 len=576 karg=ffff9ada45536000 uarg=0000000000000000 00000100:00080000:3.0:1788572688.129016:0:92463:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff9ada51140000 already not connected (conn 373, was 0): DISCONN 00000100:00080000:3.0:1788572688.129023:0:92463:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572688.129031:0:92463:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4604 00000100:00080000:3.0:1788572688.129041:0:92463:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572688.129049:0:92463:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572688.129059:0:92463:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572688.129088:0:92463:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2300 x1875449616293120/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572688.129115:0:92463:0:(recover.c:337:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery started, waiting 20 seconds 00000100:00100000:1.0:1788572688.129148:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616293120:192.168.202.113@tcp:38:lctl.0 00000800:00000100:1.0:1788572688.129219:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572688.129242:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572688/real 1788572688] req@ffff9ada604e2300 x1875449616293120/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572713 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'lctl.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572688.129255:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2300 x1875449616293120/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572713 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'lctl.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572688.129263:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572688.129266:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572688.129276:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616293120:192.168.202.113@tcp:38:lctl.0 00010000:00010000:0.0:1788572688.247048:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572689.271057:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572690.295090:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572691.319028:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788572691.831000:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572691.831014:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788572691.831020:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572691.831033:0:91222:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4604 00000100:00080000:1.0:1788572691.831048:0:91222:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572691.831058:0:91222:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788572691.831125:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f800 x1875449616293248/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572691.831156:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572691.831160:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788572691.831162:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572691.831165:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4605 00000100:00080000:1.0:1788572691.831168:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572691.831171:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572691.831181:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572691.831192:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ed80 x1875449616293376/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572691.831199:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572691.831203:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788572691.831204:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572691.831206:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4604 00000100:00080000:1.0:1788572691.831208:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572691.831214:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788572691.831216:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616293248:192.168.202.113@tcp:250:kworker.0 00000008:00100000:1.0:1788572691.831218:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572691.831228:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ea00 x1875449616293504/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788572691.831306:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572691.831353:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572691/real 1788572691] req@ffff9ada4566f800 x1875449616293248/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572716 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572691.831377:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f800 x1875449616293248/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572716 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572691.831390:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572691.831395:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572691.831399:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616293248:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572691.831408:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616293376:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572691.831432:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572691.831446:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572691/real 1788572691] req@ffff9ada4566ed80 x1875449616293376/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572716 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572691.831460:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ed80 x1875449616293376/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572716 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572691.831470:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572691.831474:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572691.831491:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616293376:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572691.831527:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616293504:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572691.831553:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572691.831577:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572691/real 1788572691] req@ffff9ada4566ea00 x1875449616293504/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572716 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572691.831592:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ea00 x1875449616293504/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572716 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572691.831602:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572691.831606:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572691.831610:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616293504:192.168.202.113@tcp:38:kworker.0 00010000:00010000:0.0:1788572692.343021:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572693.367012:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572694.390959:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572695.415083:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572696.439060:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788572696.950947:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572696.950979:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788572696.950984:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572696.950992:0:91222:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4609 00000100:00080000:3.0:1788572696.951001:0:91222:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572696.951007:0:91222:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788572696.951038:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2d80 x1875449616293632/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572696.951062:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572696.951065:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788572696.951071:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572696.951074:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4609 00000100:00080000:3.0:1788572696.951077:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572696.951080:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572696.951089:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572696.951097:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616293632:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572696.951100:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3b80 x1875449616293760/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572696.951111:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572696.951113:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788572696.951114:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572696.951117:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4609 00000100:00080000:3.0:1788572696.951119:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572696.951121:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572696.951124:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572696.951133:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0000 x1875449616293888/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788572696.951158:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572696.951188:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572696/real 1788572696] req@ffff9ada604e2d80 x1875449616293632/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572721 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572696.951202:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2d80 x1875449616293632/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572721 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572696.951210:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572696.951213:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572696.951216:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616293632:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788572696.951234:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616293760:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572696.951249:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572696.951257:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572696/real 1788572696] req@ffff9ada604e3b80 x1875449616293760/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572721 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572696.951265:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3b80 x1875449616293760/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572721 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572696.951270:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572696.951273:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572696.951283:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616293760:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572696.951288:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616293888:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572696.951300:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572696.951308:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572696/real 1788572696] req@ffff9ada604e0000 x1875449616293888/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572721 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572696.951322:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0000 x1875449616293888/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572721 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572696.951332:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572696.951336:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572696.951342:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616293888:192.168.202.113@tcp:38:kworker.0 00010000:00010000:0.0:1788572697.463065:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572698.487113:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572699.511048:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572700.535037:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572701.559072:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788572702.070937:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572702.070949:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788572702.070954:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572702.070967:0:91222:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4614 00000100:00080000:1.0:1788572702.070981:0:91222:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572702.070991:0:91222:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788572702.071042:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e300 x1875449616294016/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572702.071072:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572702.071075:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788572702.071077:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572702.071080:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4614 00000100:00080000:1.0:1788572702.071084:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572702.071089:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572702.071099:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572702.071107:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616294016:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788572702.071113:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f480 x1875449616294144/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572702.071131:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572702.071134:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788572702.071135:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572702.071138:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4614 00000100:00080000:1.0:1788572702.071141:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572702.071144:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572702.071147:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572702.071156:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d180 x1875449616294272/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788572702.071184:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572702.071230:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572702/real 1788572702] req@ffff9ada4566e300 x1875449616294016/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572727 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572702.071250:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e300 x1875449616294016/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572727 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572702.071259:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572702.071262:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572702.071266:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616294016:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572702.071289:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616294144:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572702.071307:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572702.071317:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572702/real 1788572702] req@ffff9ada4566f480 x1875449616294144/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572727 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572702.071326:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f480 x1875449616294144/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572727 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572702.071332:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572702.071334:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572702.071348:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616294144:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572702.071355:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616294272:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572702.071376:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572702.071389:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572702/real 1788572702] req@ffff9ada4566d180 x1875449616294272/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572727 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572702.071405:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d180 x1875449616294272/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572727 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572702.071412:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572702.071415:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572702.071417:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616294272:192.168.202.113@tcp:38:kworker.0 00010000:00010000:0.0:1788572702.583055:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572703.607008:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572704.630986:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572705.655049:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572706.679077:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788572707.190958:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572707.190969:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788572707.190974:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572707.190986:0:91222:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4619 00000100:00080000:3.0:1788572707.190998:0:91222:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572707.191005:0:91222:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788572707.191054:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0a80 x1875449616294400/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572707.191090:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572707.191094:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788572707.191095:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572707.191099:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4619 00000100:00080000:3.0:1788572707.191102:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572707.191106:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572707.191118:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572707.191133:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616294400:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572707.191133:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1c00 x1875449616294528/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572707.191148:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572707.191151:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788572707.191153:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572707.191156:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4619 00000100:00080000:3.0:1788572707.191158:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572707.191161:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572707.191164:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572707.191173:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3100 x1875449616294656/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788572707.191218:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572707.191255:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572707/real 1788572707] req@ffff9ada604e0a80 x1875449616294400/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572732 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572707.191271:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0a80 x1875449616294400/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572732 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572707.191281:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572707.191285:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572707.191288:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616294400:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788572707.191314:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616294528:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572707.191339:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572707.191354:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572707/real 1788572707] req@ffff9ada604e1c00 x1875449616294528/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572732 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572707.191365:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1c00 x1875449616294528/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572732 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572707.191371:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572707.191374:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572707.191388:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616294528:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572707.191392:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616294656:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572707.191412:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572707.191421:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572707/real 1788572707] req@ffff9ada604e3100 x1875449616294656/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572732 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572707.191432:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3100 x1875449616294656/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572732 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572707.191438:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572707.191440:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572707.191442:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616294656:192.168.202.113@tcp:38:kworker.0 00010000:00010000:0.0:1788572707.703040:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788572708.150959:0:92463:0:(recover.c:348:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery finished DISCONN, rc = -110 00000080:00200008:0.0:1788572708.154619:0:92467:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1788572708.154628:0:92467:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572708.154645:0:92467:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572708.154661:0:92467:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 48 lru_type: 1 00010000:00010000:0.0:1788572708.154689:0:92467:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 48 lru_type: 1 00000080:00200000:0.0:1788572708.154700:0:92467:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572708.154706:0:92467:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572708.154709:0:92467:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:0.0:1788572708.154715:0:92467:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1ce7cb8 00010000:00010000:0.0:1788572708.154718:0:92467:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 48 lru_type: 1 00010000:00010000:0.0:1788572708.154725:0:92467:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 49 lru_type: 1 00000080:00200000:0.0:1788572708.154747:0:92467:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572708.154751:0:92467:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572708.154756:0:92467:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 49 lru_type: 1 00010000:00010000:0.0:1788572708.154761:0:92467:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 49 lru_type: 1 00000080:00200000:0.0:1788572708.154766:0:92467:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572708.154768:0:92467:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572708.154770:0:92467:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:0.0:1788572708.154772:0:92467:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1ce7bb0 00010000:00010000:0.0:1788572708.154775:0:92467:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 49 lru_type: 1 00010000:00010000:0.0:1788572708.154780:0:92467:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 50 lru_type: 1 00000080:00200000:0.0:1788572708.154785:0:92467:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:0.0:1788572708.154791:0:92467:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100400 00000080:00200008:0.0:1788572708.154801:0:92467:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200008:0.0:1788572708.155333:0:92467:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1788572708.155338:0:92467:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572708.155344:0:92467:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572708.155350:0:92467:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 50 lru_type: 1 00010000:00010000:0.0:1788572708.155357:0:92467:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 50 lru_type: 1 00000080:00200000:0.0:1788572708.155362:0:92467:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572708.155365:0:92467:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572708.155368:0:92467:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:0.0:1788572708.155370:0:92467:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1ce7cb8 00010000:00010000:0.0:1788572708.155373:0:92467:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 50 lru_type: 1 00010000:00010000:0.0:1788572708.155378:0:92467:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 51 lru_type: 1 00000080:00200008:0.0:1788572708.155395:0:92467:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1788572708.155398:0:92467:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572708.155402:0:92467:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572708.155405:0:92467:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 51 lru_type: 1 00010000:00010000:0.0:1788572708.155411:0:92467:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 51 lru_type: 1 00000080:00200000:0.0:1788572708.155415:0:92467:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572708.155418:0:92467:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572708.155419:0:92467:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:0.0:1788572708.155421:0:92467:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1ce7cb8 00010000:00010000:0.0:1788572708.155423:0:92467:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 51 lru_type: 1 00010000:00010000:0.0:1788572708.155428:0:92467:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 52 lru_type: 1 00000080:00200000:0.0:1788572708.155441:0:92467:0:(llite_lib.c:2903:ll_statfs()) VFS Op:sb=lustre (ffff9ada58b0a000) / proj=0 00000080:00000004:0.0:1788572708.155444:0:92467:0:(obd_class.h:1090:obd_statfs()) lustre-clilmv-ffff9ada58b0a000: age 4179, max_age 4624 00800000:00000004:0.0:1788572708.155451:0:92467:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: age 4488, max_age 4624 00000100:00100000:0.0:1788572708.155479:0:92467:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada429ade40 going to sleep for 0 seconds 00000100:00100000:0.0:1788572708.155488:0:92467:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job df:35eb2d60-2de0-4b00-9815-d5153edd0057:92467:1875449616294784:192.168.202.113@tcp:41:df.0 00800000:00020000:0.0:1788572708.155500:0:92467:0:(lmv_obd.c:1471:lmv_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: can't stat MDS #0: rc = -110 00800000:00000004:0.0:1788572708.162084:0:92467:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: age 4488, max_age 4624 00000100:00100000:0.0:1788572708.162112:0:92467:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada429ade40 going to sleep for 0 seconds 00000100:00100000:0.0:1788572708.162120:0:92467:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job df:35eb2d60-2de0-4b00-9815-d5153edd0057:92467:1875449616294912:192.168.202.113@tcp:41:df.0 00800000:00020000:0.0:1788572708.162133:0:92467:0:(lmv_obd.c:1471:lmv_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: can't stat MDS #1: rc = -110 00010000:00010000:0.0:1788572708.727026:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572709.751032:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572710.775037:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572711.798979:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788572712.310926:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572712.310934:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788572712.310938:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572712.310946:0:91222:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4624 00000100:00080000:3.0:1788572712.310955:0:91222:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572712.310961:0:91222:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788572712.310994:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1f80 x1875449616295040/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572712.311015:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572712.311018:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788572712.311019:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572712.311022:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4624 00000100:00080000:3.0:1788572712.311025:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572712.311028:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572712.311037:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572712.311052:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2680 x1875449616295168/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572712.311063:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572712.311067:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788572712.311070:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572712.311074:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4624 00000100:00100000:1.0:1788572712.311080:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616295040:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788572712.311081:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572712.311089:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572712.311094:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572712.311106:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1500 x1875449616295296/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788572712.311152:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572712.311187:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572712/real 1788572712] req@ffff9ada604e1f80 x1875449616295040/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572737 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572712.311207:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1f80 x1875449616295040/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572737 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572712.311219:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572712.311223:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572712.311228:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616295040:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788572712.311235:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616295168:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572712.311258:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572712.311271:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572712/real 1788572712] req@ffff9ada604e2680 x1875449616295168/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572737 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572712.311284:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2680 x1875449616295168/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572737 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572712.311293:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572712.311296:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572712.311300:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616295168:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572712.311324:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616295296:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572712.311345:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572712.311357:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572712/real 1788572712] req@ffff9ada604e1500 x1875449616295296/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572737 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572712.311370:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1500 x1875449616295296/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572737 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572712.311386:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572712.311389:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572712.311392:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616295296:192.168.202.113@tcp:38:kworker.0 00000020:00000080:1.0:1788572712.704248:0:92486:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=40086685 len=0 uarg=00007ffec20ac830 00000002:00000080:1.0:1788572712.704264:0:92486:0:(mdc_request.c:2317:mdc_iocontrol()) lustre-MDT0000-mdc-ffff9ada58b0a000: cmd=40086685 len=576 karg=ffff9ada5164bc00 uarg=0000000000000000 00000100:00080000:1.0:1788572712.704269:0:92486:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff9ada51140000 already not connected (conn 379, was 0): DISCONN 00000100:00080000:1.0:1788572712.704274:0:92486:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572712.704281:0:92486:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4629 00000100:00080000:1.0:1788572712.704290:0:92486:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572712.704296:0:92486:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572712.704305:0:92486:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572712.704328:0:92486:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d500 x1875449616295424/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572712.704351:0:92486:0:(recover.c:337:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery started, waiting 20 seconds 00000100:00100000:3.0:1788572712.704392:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616295424:192.168.202.113@tcp:38:lctl.0 00000800:00000100:3.0:1788572712.704445:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572712.704466:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572712/real 1788572712] req@ffff9ada4566d500 x1875449616295424/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572737 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'lctl.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572712.704480:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d500 x1875449616295424/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572737 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'lctl.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572712.704488:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572712.704491:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572712.704502:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616295424:192.168.202.113@tcp:38:lctl.0 00010000:00010000:0.0:1788572712.823002:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572713.846986:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572714.871027:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572715.894979:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572716.919018:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788572717.430960:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572717.430976:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788572717.430982:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572717.430995:0:91222:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4629 00000100:00080000:2.0:1788572717.431009:0:91222:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572717.431017:0:91222:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788572717.431069:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616295552/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788572717.431094:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572717.431098:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788572717.431100:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572717.431104:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4630 00000100:00080000:2.0:1788572717.431109:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572717.431113:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572717.431128:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788572717.431141:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616295680/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788572717.431142:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616295552:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788572717.431152:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572717.431155:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788572717.431157:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572717.431161:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4629 00000100:00080000:2.0:1788572717.431164:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572717.431168:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572717.431172:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788572717.431181:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616295808/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788572717.431222:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572717.431259:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572717/real 1788572717] req@ffff9ada60625f80 x1875449616295552/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572742 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572717.431276:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616295552/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572742 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572717.431286:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572717.431289:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572717.431292:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616295552:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572717.431317:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616295680:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572717.431334:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572717.431343:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572717/real 1788572717] req@ffff9ada60625c00 x1875449616295680/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572742 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572717.431351:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616295680/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572742 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572717.431365:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572717.431368:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572717.431377:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616295680:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572717.431381:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616295808:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572717.431400:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572717.431413:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572717/real 1788572717] req@ffff9ada60625880 x1875449616295808/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572742 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572717.431430:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616295808/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572742 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572717.431464:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572717.431468:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572717.431470:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616295808:192.168.202.113@tcp:38:kworker.0 00010000:00010000:0.0:1788572717.943028:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572718.967061:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572719.991040:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572721.015050:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572722.038999:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788572722.550965:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572722.550977:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788572722.550981:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572722.550991:0:91222:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4635 00000100:00080000:1.0:1788572722.551003:0:91222:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572722.551011:0:91222:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788572722.551059:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c380 x1875449616295936/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572722.551092:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572722.551101:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788572722.551103:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572722.551106:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4635 00000100:00080000:1.0:1788572722.551110:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572722.551114:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572722.551125:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572722.551136:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ca80 x1875449616296064/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788572722.551142:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616295936:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788572722.551149:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572722.551153:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788572722.551155:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572722.551159:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4635 00000100:00080000:1.0:1788572722.551161:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572722.551163:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572722.551167:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572722.551175:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f100 x1875449616296192/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788572722.551256:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616296064:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572722.551283:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616296192:192.168.202.113@tcp:38:kworker.0 00010000:00010000:0.0:1788572723.063035:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572724.087139:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1788572724.983206:0:2373:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:2.0:1788572724.983294:0:2373:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1788572724.983317:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788572724.983365:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788572724.983376:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:3.0:1788572724.983413:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572722/real 1788572724] req@ffff9ada4566ca80 x1875449616296064/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572747 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572724.983455:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ca80 x1875449616296064/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572747 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572724.983469:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572724.983482:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572724.983500:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616296064:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572724.983510:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572722/real 1788572724] req@ffff9ada4566f100 x1875449616296192/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572747 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572724.983532:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f100 x1875449616296192/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572747 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572724.983540:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572724.983546:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572724.983555:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616296192:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572724.983565:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572722/real 1788572724] req@ffff9ada4566c380 x1875449616295936/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572747 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572724.983581:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c380 x1875449616295936/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572747 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572724.983590:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572724.983593:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572724.983596:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616295936:192.168.202.113@tcp:250:kworker.0 00010000:00010000:0.0:1788572725.111093:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572726.135032:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572727.159123:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788572727.670985:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572727.670998:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788572727.671002:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572727.671013:0:77466:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4640 00000100:00080000:1.0:1788572727.671024:0:77466:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572727.671032:0:77466:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788572727.671070:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c700 x1875449616296320/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572727.671101:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572727.671110:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788572727.671113:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572727.671117:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4640 00000100:00080000:1.0:1788572727.671121:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572727.671125:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572727.671135:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572727.671150:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ce00 x1875449616296448/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788572727.671153:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616296320:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788572727.671171:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572727.671175:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788572727.671177:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572727.671196:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4640 00000100:00080000:1.0:1788572727.671202:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572727.671208:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572727.671213:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572727.671237:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d880 x1875449616296576/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788572727.671239:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572727.671286:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572727/real 1788572727] req@ffff9ada4566c700 x1875449616296320/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572752 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572727.671304:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c700 x1875449616296320/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572752 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572727.671312:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572727.671316:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572727.671320:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616296320:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572727.671343:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616296448:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572727.671369:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572727.671382:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572727/real 1788572727] req@ffff9ada4566ce00 x1875449616296448/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572752 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572727.671391:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ce00 x1875449616296448/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572752 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572727.671397:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572727.671399:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572727.671412:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616296448:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572727.671416:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616296576:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572727.671432:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572727.671442:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572727/real 1788572727] req@ffff9ada4566d880 x1875449616296576/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572752 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572727.671455:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d880 x1875449616296576/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572752 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572727.671461:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572727.671464:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572727.671466:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616296576:192.168.202.113@tcp:38:kworker.0 00010000:00010000:0.0:1788572728.182993:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572729.207040:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572730.231043:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572731.255153:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572732.279112:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788572732.790979:0:92486:0:(recover.c:348:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery finished DISCONN, rc = -110 00000100:00080000:0.0:1788572732.790994:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572732.791008:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788572732.791012:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572732.791022:0:77466:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4645 00000100:00080000:0.0:1788572732.791032:0:77466:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572732.791039:0:77466:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788572732.791091:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616296704/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788572732.791114:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572732.791117:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788572732.791119:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572732.791122:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4645 00000100:00080000:0.0:1788572732.791125:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572732.791128:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788572732.791137:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788572732.791151:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f100 x1875449616296832/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788572732.791162:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572732.791166:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788572732.791168:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572732.791175:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4645 00000100:00100000:3.0:1788572732.791175:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616296704:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788572732.791180:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572732.791186:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788572732.791192:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788572732.791208:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557df80 x1875449616296960/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788572732.791253:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572732.791306:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572732/real 1788572732] req@ffff9ada4557ed80 x1875449616296704/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572757 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572732.791332:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616296704/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572757 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572732.791345:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572732.791349:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572732.791354:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616296704:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572732.791363:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616296832:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572732.791394:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572732.791410:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572732/real 1788572732] req@ffff9ada4557f100 x1875449616296832/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572757 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572732.791427:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f100 x1875449616296832/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572757 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572732.791446:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572732.791450:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572732.791455:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616296832:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572732.791489:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616296960:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572732.791523:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572732.791538:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572732/real 1788572732] req@ffff9ada4557df80 x1875449616296960/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572757 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572732.791555:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557df80 x1875449616296960/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572757 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572732.791565:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572732.791570:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572732.791575:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616296960:192.168.202.113@tcp:38:kworker.0 00000080:00200008:0.0:1788572732.794765:0:92490:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1788572732.794774:0:92490:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572732.794791:0:92490:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572732.794807:0:92490:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 52 lru_type: 1 00010000:00010000:0.0:1788572732.794817:0:92490:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 52 lru_type: 1 00000080:00200000:0.0:1788572732.794824:0:92490:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572732.794830:0:92490:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572732.794833:0:92490:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:0.0:1788572732.794839:0:92490:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1e1fcb8 00010000:00010000:0.0:1788572732.794842:0:92490:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 52 lru_type: 1 00010000:00010000:0.0:1788572732.794850:0:92490:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 53 lru_type: 1 00000080:00200000:0.0:1788572732.794892:0:92490:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572732.794897:0:92490:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572732.794903:0:92490:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 53 lru_type: 1 00010000:00010000:0.0:1788572732.794909:0:92490:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 53 lru_type: 1 00000080:00200000:0.0:1788572732.794914:0:92490:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572732.794917:0:92490:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572732.794919:0:92490:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:0.0:1788572732.794922:0:92490:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1e1fbb0 00010000:00010000:0.0:1788572732.794925:0:92490:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 53 lru_type: 1 00010000:00010000:0.0:1788572732.794930:0:92490:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 54 lru_type: 1 00000080:00200000:0.0:1788572732.794936:0:92490:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:0.0:1788572732.794942:0:92490:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100400 00000080:00200008:0.0:1788572732.794954:0:92490:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200008:0.0:1788572732.795721:0:92490:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1788572732.795728:0:92490:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572732.795740:0:92490:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572732.795750:0:92490:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 54 lru_type: 1 00010000:00010000:0.0:1788572732.795761:0:92490:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 54 lru_type: 1 00000080:00200000:0.0:1788572732.795768:0:92490:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572732.795773:0:92490:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572732.795777:0:92490:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:0.0:1788572732.795783:0:92490:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1e1fcb8 00010000:00010000:0.0:1788572732.795787:0:92490:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 54 lru_type: 1 00010000:00010000:0.0:1788572732.795796:0:92490:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 55 lru_type: 1 00000080:00200008:0.0:1788572732.795825:0:92490:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1788572732.795829:0:92490:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572732.795836:0:92490:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572732.795844:0:92490:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 55 lru_type: 1 00010000:00010000:0.0:1788572732.795977:0:92490:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 55 lru_type: 1 00000080:00200000:0.0:1788572732.795985:0:92490:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572732.795990:0:92490:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572732.795993:0:92490:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:0.0:1788572732.795997:0:92490:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1e1fcb8 00010000:00010000:0.0:1788572732.796001:0:92490:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 55 lru_type: 1 00010000:00010000:0.0:1788572732.796010:0:92490:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 56 lru_type: 1 00000080:00200000:0.0:1788572732.796034:0:92490:0:(llite_lib.c:2903:ll_statfs()) VFS Op:sb=lustre (ffff9ada58b0a000) / proj=0 00000080:00000004:0.0:1788572732.796039:0:92490:0:(obd_class.h:1090:obd_statfs()) lustre-clilmv-ffff9ada58b0a000: age 4179, max_age 4649 00800000:00000004:0.0:1788572732.796051:0:92490:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: age 4488, max_age 4649 00000100:00100000:0.0:1788572732.796099:0:92490:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada429ade40 going to sleep for 0 seconds 00000100:00100000:0.0:1788572732.796108:0:92490:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job df:35eb2d60-2de0-4b00-9815-d5153edd0057:92490:1875449616297088:192.168.202.113@tcp:41:df.0 00800000:00020000:0.0:1788572732.796121:0:92490:0:(lmv_obd.c:1471:lmv_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: can't stat MDS #0: rc = -110 00800000:00000004:0.0:1788572732.803142:0:92490:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: age 4488, max_age 4649 00000100:00100000:0.0:1788572732.803194:0:92490:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada429ade40 going to sleep for 0 seconds 00000100:00100000:0.0:1788572732.803205:0:92490:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job df:35eb2d60-2de0-4b00-9815-d5153edd0057:92490:1875449616297216:192.168.202.113@tcp:41:df.0 00800000:00020000:0.0:1788572732.803221:0:92490:0:(lmv_obd.c:1471:lmv_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: can't stat MDS #1: rc = -110 00010000:00010000:0.0:1788572733.303099:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572734.327033:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572735.351014:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572736.375044:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:3.0:1788572737.280392:0:92508:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=40086685 len=0 uarg=00007ffe7aa99f90 00000002:00000080:3.0:1788572737.280416:0:92508:0:(mdc_request.c:2317:mdc_iocontrol()) lustre-MDT0000-mdc-ffff9ada58b0a000: cmd=40086685 len=576 karg=ffff9ada45534800 uarg=0000000000000000 00000100:00080000:3.0:1788572737.280426:0:92508:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff9ada51140000 already not connected (conn 384, was 0): DISCONN 00000100:00080000:3.0:1788572737.280434:0:92508:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572737.280443:0:92508:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4650 00000100:00080000:3.0:1788572737.280455:0:92508:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572737.280462:0:92508:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572737.280474:0:92508:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572737.280506:0:92508:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0e00 x1875449616297344/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572737.280530:0:92508:0:(recover.c:337:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery started, waiting 20 seconds 00000100:00100000:1.0:1788572737.280580:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616297344:192.168.202.113@tcp:38:lctl.0 00000800:00000100:1.0:1788572737.280659:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572737.280688:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572737/real 1788572737] req@ffff9ada604e0e00 x1875449616297344/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572762 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'lctl.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572737.280706:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0e00 x1875449616297344/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572762 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'lctl.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572737.280715:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572737.280718:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572737.280730:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616297344:192.168.202.113@tcp:38:lctl.0 00010000:00010000:3.0:1788572737.399136:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788572737.910946:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572737.910954:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788572737.910957:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572737.910966:0:77466:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4650 00000100:00080000:1.0:1788572737.910975:0:77466:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572737.910981:0:77466:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788572737.911016:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e680 x1875449616297472/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572737.911041:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572737.911044:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788572737.911045:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572737.911049:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4654 00000100:00080000:1.0:1788572737.911051:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572737.911053:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572737.911061:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572737.911071:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d880 x1875449616297600/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572737.911084:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572737.911086:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788572737.911087:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572737.911090:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4650 00000100:00080000:1.0:1788572737.911092:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572737.911095:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572737.911100:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572737.911101:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616297472:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788572737.911109:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ce00 x1875449616297728/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788572737.911159:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572737.911187:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572737/real 1788572737] req@ffff9ada4566e680 x1875449616297472/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572762 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572737.911202:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e680 x1875449616297472/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572762 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572737.911210:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572737.911213:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572737.911216:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616297472:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572737.911236:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616297600:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572737.911256:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572737.911271:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572737/real 1788572737] req@ffff9ada4566d880 x1875449616297600/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572762 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572737.911285:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d880 x1875449616297600/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572762 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572737.911294:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572737.911297:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572737.911310:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616297600:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572737.911320:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616297728:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572737.911343:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572737.911355:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572737/real 1788572737] req@ffff9ada4566ce00 x1875449616297728/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572762 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572737.911369:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ce00 x1875449616297728/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572762 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572737.911376:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572737.911379:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572737.911381:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616297728:192.168.202.113@tcp:38:kworker.0 00010000:00010000:0.0:1788572738.423019:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572739.447056:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572740.471065:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572741.495089:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572742.519035:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788572743.030985:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572743.030994:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788572743.030998:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572743.031007:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4655 00000100:00080000:3.0:1788572743.031020:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572743.031027:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788572743.031061:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2a00 x1875449616297856/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572743.031084:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572743.031087:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788572743.031089:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572743.031092:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4655 00000100:00080000:3.0:1788572743.031094:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572743.031099:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572743.031107:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572743.031118:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616297856:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572743.031119:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0e00 x1875449616297984/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572743.031136:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572743.031139:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788572743.031140:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572743.031143:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4655 00000100:00080000:3.0:1788572743.031146:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572743.031148:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572743.031151:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572743.031159:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1500 x1875449616298112/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788572743.031181:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572743.031216:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572743/real 1788572743] req@ffff9ada604e2a00 x1875449616297856/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572768 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572743.031235:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2a00 x1875449616297856/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572768 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572743.031254:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572743.031259:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572743.031263:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616297856:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788572743.031288:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616297984:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572743.031314:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572743.031328:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572743/real 1788572743] req@ffff9ada604e0e00 x1875449616297984/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572768 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572743.031342:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0e00 x1875449616297984/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572768 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572743.031351:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572743.031355:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572743.031369:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616297984:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572743.031376:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616298112:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572743.031398:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572743.031414:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572743/real 1788572743] req@ffff9ada604e1500 x1875449616298112/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572768 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572743.031428:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1500 x1875449616298112/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572768 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572743.031436:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572743.031439:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572743.031442:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616298112:192.168.202.113@tcp:38:kworker.0 00010000:00010000:3.0:1788572743.542999:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572744.567064:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572745.591024:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572746.615061:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572747.639023:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788572748.150982:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572748.150990:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788572748.150994:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572748.151003:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4660 00000100:00080000:0.0:1788572748.151012:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572748.151018:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788572748.151064:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d500 x1875449616298240/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788572748.151088:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572748.151091:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788572748.151093:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572748.151096:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4660 00000100:00080000:0.0:1788572748.151098:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572748.151100:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788572748.151108:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788572748.151118:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616298368/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788572748.151126:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572748.151128:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788572748.151130:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1788572748.151134:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616298240:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788572748.151136:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4660 00000100:00080000:0.0:1788572748.151143:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572748.151147:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788572748.151150:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788572748.151166:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557df80 x1875449616298496/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788572748.151206:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572748.151244:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572748/real 1788572748] req@ffff9ada4557d500 x1875449616298240/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572773 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572748.151263:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d500 x1875449616298240/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572773 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572748.151272:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572748.151275:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572748.151278:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616298240:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788572748.151297:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616298368:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572748.151315:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572748.151324:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572748/real 1788572748] req@ffff9ada4557ea00 x1875449616298368/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572773 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572748.151344:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616298368/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572773 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572748.151350:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572748.151352:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572748.151361:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616298368:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572748.151366:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616298496:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572748.151381:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572748.151390:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572748/real 1788572748] req@ffff9ada4557df80 x1875449616298496/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572773 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572748.151398:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557df80 x1875449616298496/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572773 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572748.151404:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572748.151407:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572748.151409:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616298496:192.168.202.113@tcp:38:kworker.0 00010000:00010000:0.0:1788572748.662985:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572749.686977:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572750.711038:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572751.735017:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572752.759046:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788572753.270968:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572753.270977:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788572753.270980:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572753.270989:0:77466:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4665 00000100:00080000:2.0:1788572753.270998:0:77466:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572753.271004:0:77466:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788572753.271038:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616298624/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788572753.271072:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572753.271076:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788572753.271078:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572753.271082:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4665 00000100:00080000:2.0:1788572753.271086:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572753.271091:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572753.271101:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788572753.271114:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616298752/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788572753.271126:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572753.271129:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788572753.271131:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572753.271136:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4665 00000100:00100000:1.0:1788572753.271137:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616298624:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788572753.271139:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572753.271145:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572753.271149:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788572753.271160:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616298880/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788572753.271196:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572753.271225:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572753/real 1788572753] req@ffff9ada60624a80 x1875449616298624/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572778 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572753.271238:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616298624/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572778 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572753.271257:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572753.271260:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572753.271263:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616298624:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788572753.271269:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616298752:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572753.271284:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572753.271292:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572753/real 1788572753] req@ffff9ada60626a00 x1875449616298752/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572778 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572753.271300:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616298752/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572778 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572753.271306:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572753.271308:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572753.271320:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616298752:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572753.271339:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616298880:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572753.271354:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572753.271364:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572753/real 1788572753] req@ffff9ada60624380 x1875449616298880/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572778 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572753.271373:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616298880/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572778 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572753.271379:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572753.271381:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572753.271384:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616298880:192.168.202.113@tcp:38:kworker.0 00010000:00010000:0.0:1788572753.783038:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572754.806986:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572755.831075:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572756.855044:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788572757.303005:0:92508:0:(recover.c:348:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery finished DISCONN, rc = -110 00000080:00200008:0.0:1788572757.307949:0:92512:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1788572757.307964:0:92512:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572757.307994:0:92512:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572757.308018:0:92512:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 56 lru_type: 1 00010000:00010000:0.0:1788572757.308038:0:92512:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 56 lru_type: 1 00000080:00200000:0.0:1788572757.308049:0:92512:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572757.308058:0:92512:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572757.308062:0:92512:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:0.0:1788572757.308072:0:92512:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1e17cb8 00010000:00010000:0.0:1788572757.308078:0:92512:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 56 lru_type: 1 00010000:00010000:0.0:1788572757.308089:0:92512:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 57 lru_type: 1 00000080:00200000:0.0:1788572757.308123:0:92512:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572757.308131:0:92512:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572757.308139:0:92512:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 57 lru_type: 1 00010000:00010000:0.0:1788572757.308149:0:92512:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 57 lru_type: 1 00000080:00200000:0.0:1788572757.308157:0:92512:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572757.308162:0:92512:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572757.308165:0:92512:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:0.0:1788572757.308169:0:92512:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1e17bb0 00010000:00010000:0.0:1788572757.308173:0:92512:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 57 lru_type: 1 00010000:00010000:0.0:1788572757.308182:0:92512:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 58 lru_type: 1 00000080:00200000:0.0:1788572757.308191:0:92512:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:0.0:1788572757.308200:0:92512:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100400 00000080:00200008:0.0:1788572757.308216:0:92512:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200008:0.0:1788572757.309190:0:92512:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1788572757.309197:0:92512:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572757.309206:0:92512:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572757.309216:0:92512:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 58 lru_type: 1 00010000:00010000:0.0:1788572757.309227:0:92512:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 58 lru_type: 1 00000080:00200000:0.0:1788572757.309235:0:92512:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572757.309240:0:92512:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572757.309244:0:92512:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:0.0:1788572757.309249:0:92512:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1e17cb8 00010000:00010000:0.0:1788572757.309253:0:92512:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 58 lru_type: 1 00010000:00010000:0.0:1788572757.309262:0:92512:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 59 lru_type: 1 00000080:00200008:0.0:1788572757.309290:0:92512:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1788572757.309295:0:92512:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572757.309302:0:92512:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572757.309310:0:92512:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 59 lru_type: 1 00010000:00010000:0.0:1788572757.309320:0:92512:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 59 lru_type: 1 00000080:00200000:0.0:1788572757.309328:0:92512:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572757.309333:0:92512:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572757.309336:0:92512:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:0.0:1788572757.309341:0:92512:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1e17cb8 00010000:00010000:0.0:1788572757.309345:0:92512:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 59 lru_type: 1 00010000:00010000:0.0:1788572757.309355:0:92512:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 60 lru_type: 1 00000080:00200000:0.0:1788572757.309380:0:92512:0:(llite_lib.c:2903:ll_statfs()) VFS Op:sb=lustre (ffff9ada58b0a000) / proj=0 00000080:00000004:0.0:1788572757.309385:0:92512:0:(obd_class.h:1090:obd_statfs()) lustre-clilmv-ffff9ada58b0a000: age 4179, max_age 4673 00800000:00000004:0.0:1788572757.309399:0:92512:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: age 4488, max_age 4673 00000100:00100000:0.0:1788572757.309445:0:92512:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada429ade40 going to sleep for 0 seconds 00000100:00100000:0.0:1788572757.309460:0:92512:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f100 pname:cluuid:pid:xid:nid:opc:job df:35eb2d60-2de0-4b00-9815-d5153edd0057:92512:1875449616299008:192.168.202.113@tcp:41:df.0 00800000:00020000:0.0:1788572757.309479:0:92512:0:(lmv_obd.c:1471:lmv_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: can't stat MDS #0: rc = -110 00800000:00000004:0.0:1788572757.315786:0:92512:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: age 4488, max_age 4673 00000100:00100000:0.0:1788572757.315827:0:92512:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada429ade40 going to sleep for 0 seconds 00000100:00100000:0.0:1788572757.315838:0:92512:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f100 pname:cluuid:pid:xid:nid:opc:job df:35eb2d60-2de0-4b00-9815-d5153edd0057:92512:1875449616299136:192.168.202.113@tcp:41:df.0 00800000:00020000:0.0:1788572757.315872:0:92512:0:(lmv_obd.c:1471:lmv_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: can't stat MDS #1: rc = -110 00010000:00010000:0.0:1788572757.879017:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788572758.390969:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572758.390980:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788572758.390985:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572758.390996:0:77466:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4670 00000100:00080000:3.0:1788572758.391007:0:77466:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572758.391016:0:77466:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788572758.391058:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2680 x1875449616299264/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572758.391099:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572758.391104:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788572758.391107:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572758.391113:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4670 00000100:00080000:3.0:1788572758.391119:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572758.391124:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572758.391137:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572758.391155:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616299264:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572758.391157:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1f80 x1875449616299392/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572758.391174:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572758.391178:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788572758.391180:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572758.391195:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4670 00000100:00080000:3.0:1788572758.391204:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572758.391209:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572758.391216:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000800:00000100:1.0:1788572758.391224:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572758.391232:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3100 x1875449616299520/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788572758.391280:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572758/real 1788572758] req@ffff9ada604e2680 x1875449616299264/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572783 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572758.391302:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2680 x1875449616299264/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572783 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572758.391314:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572758.391318:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572758.391322:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616299264:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788572758.391358:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616299392:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572758.391394:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572758.391408:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572758/real 1788572758] req@ffff9ada604e1f80 x1875449616299392/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572783 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572758.391429:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1f80 x1875449616299392/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572783 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572758.391444:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572758.391447:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572758.391450:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616299392:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572758.391456:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616299520:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572758.391477:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572758.391488:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572758/real 1788572758] req@ffff9ada604e3100 x1875449616299520/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572783 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572758.391500:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3100 x1875449616299520/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572783 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572758.391507:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572758.391511:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572758.391514:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616299520:192.168.202.113@tcp:38:kworker.0 00010000:00010000:0.0:1788572758.903035:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572759.927002:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572760.951047:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572761.975079:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572762.999070:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788572763.511180:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572763.511194:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788572763.511199:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572763.511210:0:77466:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4676 00000100:00080000:0.0:1788572763.511220:0:77466:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572763.511229:0:77466:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788572763.511273:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f100 x1875449616299648/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788572763.511299:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572763.511303:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788572763.511306:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572763.511311:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4676 00000100:00080000:0.0:1788572763.511318:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572763.511323:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788572763.511333:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788572763.511350:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616299776/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788572763.511362:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572763.511367:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:1.0:1788572763.511370:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616299648:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788572763.511371:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572763.511381:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4676 00000100:00080000:0.0:1788572763.511385:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572763.511388:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788572763.511393:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788572763.511407:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d180 x1875449616299904/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788572763.511447:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572763.511487:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572763/real 1788572763] req@ffff9ada4557f100 x1875449616299648/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572763.511508:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f100 x1875449616299648/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572763.511522:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572763.511527:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572763.511532:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616299648:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788572763.511565:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616299776:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572763.511593:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572763.511611:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572763/real 1788572763] req@ffff9ada4557ed80 x1875449616299776/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572763.511627:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616299776/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572763.511636:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572763.511641:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572763.511645:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616299776:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572763.511655:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616299904:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572763.511677:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572763.511689:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572763/real 1788572763] req@ffff9ada4557d180 x1875449616299904/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572763.511702:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d180 x1875449616299904/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572763.511729:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572763.511733:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572763.511737:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616299904:192.168.202.113@tcp:38:kworker.0 00000020:00000080:3.0:1788572763.521959:0:92531:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=40086685 len=0 uarg=00007ffdf1afe150 00000002:00000080:3.0:1788572763.521983:0:92531:0:(mdc_request.c:2317:mdc_iocontrol()) lustre-MDT0000-mdc-ffff9ada58b0a000: cmd=40086685 len=576 karg=ffff9ada45534400 uarg=0000000000000000 00000100:00080000:3.0:1788572763.521991:0:92531:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff9ada51140000 already not connected (conn 391, was 0): DISCONN 00000100:00080000:3.0:1788572763.521998:0:92531:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572763.522007:0:92531:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4681 00000100:00080000:3.0:1788572763.522019:0:92531:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572763.522026:0:92531:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572763.522039:0:92531:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572763.522070:0:92531:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1c00 x1875449616300032/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572763.522095:0:92531:0:(recover.c:337:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery started, waiting 20 seconds 00000100:00100000:1.0:1788572763.522148:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616300032:192.168.202.113@tcp:38:lctl.0 00000800:00000100:1.0:1788572763.522222:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572763.522255:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572763/real 1788572763] req@ffff9ada604e1c00 x1875449616300032/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'lctl.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572763.522276:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1c00 x1875449616300032/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'lctl.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572763.522290:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572763.522295:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572763.522312:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616300032:192.168.202.113@tcp:38:lctl.0 00010000:00010000:0.0:1788572764.023063:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572765.047087:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572766.070984:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572767.095054:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572768.119041:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788572768.631001:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572768.631012:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788572768.631016:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572768.631027:0:77466:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4681 00000100:00080000:0.0:1788572768.631039:0:77466:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572768.631047:0:77466:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788572768.631092:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616300160/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788572768.631120:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572768.631124:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788572768.631126:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572768.631130:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4681 00000100:00080000:0.0:1788572768.631134:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572768.631138:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788572768.631150:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788572768.631163:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616300288/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788572768.631173:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572768.631176:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788572768.631178:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1788572768.631180:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616300160:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788572768.631182:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4681 00000100:00080000:0.0:1788572768.631186:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572768.631189:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788572768.631193:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788572768.631203:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616300416/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788572768.631274:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572768.631314:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572768/real 1788572768] req@ffff9ada4557e680 x1875449616300160/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572768.631330:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616300160/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572768.631339:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572768.631342:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572768.631345:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616300160:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788572768.631368:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616300288:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572768.631386:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572768.631394:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572768/real 1788572768] req@ffff9ada4557e300 x1875449616300288/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572768.631402:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616300288/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572768.631412:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572768.631415:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572768.631427:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616300288:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572768.631431:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616300416:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572768.631444:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572768.631451:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572768/real 1788572768] req@ffff9ada4557dc00 x1875449616300416/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572768.631461:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616300416/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572768.631467:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572768.631470:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572768.631472:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616300416:192.168.202.113@tcp:38:kworker.0 00010000:00010000:2.0:1788572769.142974:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572770.167044:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572771.191027:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572772.215009:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572773.239078:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788572773.751023:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572773.751038:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788572773.751044:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572773.751058:0:77466:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4686 00000100:00080000:2.0:1788572773.751073:0:77466:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572773.751082:0:77466:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788572773.751133:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616300544/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788572773.751165:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572773.751169:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788572773.751171:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572773.751176:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4686 00000100:00080000:2.0:1788572773.751180:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572773.751184:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572773.751197:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788572773.751209:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616300672/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788572773.751217:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572773.751220:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788572773.751222:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572773.751226:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4686 00000100:00080000:2.0:1788572773.751228:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572773.751231:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572773.751234:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788572773.751243:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627100 x1875449616300800/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788572773.751263:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616300544:192.168.202.113@tcp:250:kworker.0 00000800:00000100:1.0:1788572773.751344:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572773.751389:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572773/real 1788572773] req@ffff9ada60625500 x1875449616300544/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572798 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572773.751410:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616300544/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572798 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572773.751422:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572773.751428:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572773.751433:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616300544:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788572773.751442:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616300672:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572773.751466:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572773.751479:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572773/real 1788572773] req@ffff9ada60626680 x1875449616300672/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572798 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572773.751494:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616300672/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572798 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572773.751503:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572773.751507:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572773.751527:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616300672:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572773.751556:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616300800:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572773.751582:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572773.751596:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572773/real 1788572773] req@ffff9ada60627100 x1875449616300800/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572798 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572773.751610:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627100 x1875449616300800/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572798 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572773.751619:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572773.751622:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572773.751625:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616300800:192.168.202.113@tcp:38:kworker.0 00010000:00010000:3.0:1788572774.263038:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572775.287003:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572776.311035:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572777.335071:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572778.359019:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788572778.870958:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572778.870966:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788572778.870969:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572778.870978:0:77466:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4691 00000100:00080000:0.0:1788572778.870986:0:77466:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572778.870992:0:77466:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788572778.871026:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616300928/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788572778.871055:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572778.871058:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788572778.871059:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572778.871062:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4691 00000100:00080000:0.0:1788572778.871064:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572778.871066:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788572778.871074:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788572778.871082:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616301056/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788572778.871090:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572778.871093:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788572778.871095:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1788572778.871100:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616300928:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788572778.871101:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4691 00000100:00080000:0.0:1788572778.871106:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572778.871112:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788572778.871115:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788572778.871124:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616301184/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788572778.871170:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572778.871200:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572778/real 1788572778] req@ffff9ada4557fb80 x1875449616300928/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572803 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572778.871213:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616300928/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572803 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572778.871220:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572778.871223:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572778.871226:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616300928:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788572778.871244:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616301056:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572778.871260:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572778.871268:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572778/real 1788572778] req@ffff9ada4557d880 x1875449616301056/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572803 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572778.871276:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616301056/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572803 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572778.871282:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572778.871284:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572778.871296:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616301056:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572778.871301:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616301184:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572778.871313:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572778.871321:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572778/real 1788572778] req@ffff9ada4557f800 x1875449616301184/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572803 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572778.871338:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616301184/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572803 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572778.871344:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572778.871348:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572778.871351:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616301184:192.168.202.113@tcp:38:kworker.0 00010000:00010000:3.0:1788572779.383175:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572780.407011:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572781.431021:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572782.454986:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572783.479072:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788572783.990978:0:92531:0:(recover.c:348:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery finished DISCONN, rc = -110 00000100:00080000:2.0:1788572783.991012:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572783.991017:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788572783.991020:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572783.991030:0:77466:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4696 00000100:00080000:2.0:1788572783.991047:0:77466:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572783.991055:0:77466:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788572783.991089:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624700 x1875449616301312/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788572783.991112:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572783.991115:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788572783.991116:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572783.991119:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4696 00000100:00080000:2.0:1788572783.991121:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572783.991123:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572783.991130:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788572783.991140:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627100 x1875449616301440/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788572783.991147:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572783.991150:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788572783.991151:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572783.991154:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4696 00000100:00080000:2.0:1788572783.991157:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572783.991160:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572783.991166:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572783.991169:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616301312:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788572783.991177:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616301568/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788572783.991234:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572783.991276:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572783/real 1788572783] req@ffff9ada60624700 x1875449616301312/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572808 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572783.991299:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624700 x1875449616301312/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572808 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572783.991310:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572783.991314:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572783.991318:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616301312:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788572783.991325:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616301440:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572783.991348:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572783.991361:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572783/real 1788572783] req@ffff9ada60627100 x1875449616301440/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572808 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572783.991373:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627100 x1875449616301440/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572808 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572783.991387:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572783.991391:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572783.991394:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616301440:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572783.991416:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616301568:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572783.991433:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572783.991443:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572783/real 1788572783] req@ffff9ada60626680 x1875449616301568/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572808 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572783.991455:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616301568/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572808 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572783.991463:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572783.991466:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572783.991470:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616301568:192.168.202.113@tcp:38:kworker.0 00000080:00200008:3.0:1788572783.994932:0:92535:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1788572783.994940:0:92535:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:3.0:1788572783.994957:0:92535:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1788572783.994973:0:92535:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 60 lru_type: 1 00010000:00010000:3.0:1788572783.994983:0:92535:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 60 lru_type: 1 00000080:00200000:3.0:1788572783.994990:0:92535:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1788572783.994996:0:92535:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:3.0:1788572783.994999:0:92535:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:3.0:1788572783.995005:0:92535:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1e67cb8 00010000:00010000:3.0:1788572783.995008:0:92535:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 60 lru_type: 1 00010000:00010000:3.0:1788572783.995014:0:92535:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 61 lru_type: 1 00000080:00200000:3.0:1788572783.995034:0:92535:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:3.0:1788572783.995040:0:92535:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1788572783.995046:0:92535:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 61 lru_type: 1 00010000:00010000:3.0:1788572783.995055:0:92535:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 61 lru_type: 1 00000080:00200000:3.0:1788572783.995062:0:92535:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1788572783.995067:0:92535:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:3.0:1788572783.995070:0:92535:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:3.0:1788572783.995074:0:92535:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1e67bb0 00010000:00010000:3.0:1788572783.995078:0:92535:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 61 lru_type: 1 00010000:00010000:3.0:1788572783.995087:0:92535:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 62 lru_type: 1 00000080:00200000:3.0:1788572783.995095:0:92535:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:3.0:1788572783.995100:0:92535:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100400 00000080:00200008:3.0:1788572783.995111:0:92535:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200008:3.0:1788572783.995818:0:92535:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1788572783.995824:0:92535:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:3.0:1788572783.995834:0:92535:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1788572783.995842:0:92535:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 62 lru_type: 1 00010000:00010000:3.0:1788572783.995886:0:92535:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 62 lru_type: 1 00000080:00200000:3.0:1788572783.995897:0:92535:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1788572783.995903:0:92535:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:3.0:1788572783.995908:0:92535:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:3.0:1788572783.995914:0:92535:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1e67cb8 00010000:00010000:3.0:1788572783.995919:0:92535:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 62 lru_type: 1 00010000:00010000:3.0:1788572783.995929:0:92535:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 63 lru_type: 1 00000080:00200008:3.0:1788572783.995957:0:92535:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1788572783.995962:0:92535:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:3.0:1788572783.995970:0:92535:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1788572783.995977:0:92535:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 63 lru_type: 1 00010000:00010000:3.0:1788572783.995986:0:92535:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 63 lru_type: 1 00000080:00200000:3.0:1788572783.995999:0:92535:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1788572783.996003:0:92535:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:3.0:1788572783.996006:0:92535:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:3.0:1788572783.996010:0:92535:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1e67cb8 00010000:00010000:3.0:1788572783.996014:0:92535:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 63 lru_type: 1 00010000:00010000:3.0:1788572783.996022:0:92535:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 64 lru_type: 1 00000080:00200000:3.0:1788572783.996043:0:92535:0:(llite_lib.c:2903:ll_statfs()) VFS Op:sb=lustre (ffff9ada58b0a000) / proj=0 00000080:00000004:3.0:1788572783.996048:0:92535:0:(obd_class.h:1090:obd_statfs()) lustre-clilmv-ffff9ada58b0a000: age 4179, max_age 4700 00800000:00000004:3.0:1788572783.996057:0:92535:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: age 4488, max_age 4700 00000100:00100000:3.0:1788572783.996081:0:92535:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6059b9c0 going to sleep for 0 seconds 00000100:00100000:3.0:1788572783.996088:0:92535:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job df:35eb2d60-2de0-4b00-9815-d5153edd0057:92535:1875449616301696:192.168.202.113@tcp:41:df.0 00800000:00020000:3.0:1788572783.996100:0:92535:0:(lmv_obd.c:1471:lmv_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: can't stat MDS #0: rc = -110 00800000:00000004:3.0:1788572783.996103:0:92535:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: age 4488, max_age 4700 00000100:00100000:3.0:1788572783.996115:0:92535:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6059b9c0 going to sleep for 0 seconds 00000100:00100000:3.0:1788572783.996120:0:92535:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job df:35eb2d60-2de0-4b00-9815-d5153edd0057:92535:1875449616301824:192.168.202.113@tcp:41:df.0 00800000:00020000:3.0:1788572783.996127:0:92535:0:(lmv_obd.c:1471:lmv_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: can't stat MDS #1: rc = -110 00010000:00010000:0.0:1788572784.503025:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572785.527045:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572786.551040:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572787.575023:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572788.599037:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788572789.111095:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572789.111106:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788572789.111110:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572789.111121:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4701 00000100:00080000:3.0:1788572789.111133:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572789.111142:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788572789.111183:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0a80 x1875449616301952/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572789.111220:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572789.111224:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788572789.111226:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572789.111230:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4701 00000100:00080000:3.0:1788572789.111235:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572789.111239:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572789.111264:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572789.111275:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616301952:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572789.111282:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0000 x1875449616302080/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572789.111296:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572789.111300:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788572789.111302:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572789.111307:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4701 00000100:00080000:3.0:1788572789.111311:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572789.111315:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572789.111321:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572789.111336:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3b80 x1875449616302208/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788572789.111373:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616302080:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572789.111397:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616302208:192.168.202.113@tcp:38:kworker.0 00010000:00010000:3.0:1788572789.623028:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1788572790.199164:0:2371:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:0.0:1788572790.199213:0:2371:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1788572790.199230:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:0.0:1788572790.199272:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:0.0:1788572790.199283:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:1.0:1788572790.199329:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572789/real 1788572790] req@ffff9ada604e0000 x1875449616302080/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572814 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572790.199357:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0000 x1875449616302080/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572814 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572790.199371:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572790.199377:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572790.199381:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616302080:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572790.199391:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572789/real 1788572790] req@ffff9ada604e3b80 x1875449616302208/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572814 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572790.199406:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3b80 x1875449616302208/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572814 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572790.199415:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572790.199420:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572790.199423:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616302208:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572790.199432:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572789/real 1788572790] req@ffff9ada604e0a80 x1875449616301952/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572814 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572790.199451:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0a80 x1875449616301952/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572814 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572790.199459:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572790.199463:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572790.199467:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616301952:192.168.202.113@tcp:250:kworker.0 00000020:00000080:3.0:1788572790.208423:0:92554:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=40086685 len=0 uarg=00007ffe42ebb3c0 00000002:00000080:3.0:1788572790.208439:0:92554:0:(mdc_request.c:2317:mdc_iocontrol()) lustre-MDT0000-mdc-ffff9ada58b0a000: cmd=40086685 len=576 karg=ffff9ada45536c00 uarg=0000000000000000 00000100:00080000:3.0:1788572790.208445:0:92554:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff9ada51140000 already not connected (conn 397, was 0): DISCONN 00000100:00080000:3.0:1788572790.208451:0:92554:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572790.208458:0:92554:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4706 00000100:00080000:3.0:1788572790.208467:0:92554:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572790.208473:0:92554:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572790.208481:0:92554:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572790.208505:0:92554:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2d80 x1875449616302336/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572790.208526:0:92554:0:(recover.c:337:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery started, waiting 20 seconds 00000100:00100000:1.0:1788572790.208570:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616302336:192.168.202.113@tcp:38:lctl.0 00000800:00000100:1.0:1788572790.208623:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572790.208645:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572790/real 1788572790] req@ffff9ada604e2d80 x1875449616302336/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572815 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'lctl.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572790.208659:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2d80 x1875449616302336/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572815 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'lctl.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572790.208667:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572790.208670:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572790.208681:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616302336:192.168.202.113@tcp:38:lctl.0 00010000:00010000:0.0:1788572790.647015:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572791.671007:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572792.695018:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572793.719048:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788572794.230972:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572794.230983:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788572794.230988:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572794.230997:0:77466:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4706 00000100:00080000:0.0:1788572794.231007:0:77466:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572794.231013:0:77466:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788572794.231044:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616302464/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788572794.231066:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572794.231069:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788572794.231070:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572794.231073:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4707 00000100:00080000:0.0:1788572794.231076:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572794.231078:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788572794.231085:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788572794.231093:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c000 x1875449616302592/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788572794.231102:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572794.231106:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:1.0:1788572794.231109:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616302464:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788572794.231110:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572794.231120:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4706 00000100:00080000:0.0:1788572794.231125:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572794.231129:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788572794.231135:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788572794.231157:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616302720/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788572794.231170:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572794.231203:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572794/real 1788572794] req@ffff9ada4557ca80 x1875449616302464/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572819 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572794.231217:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616302464/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572819 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572794.231225:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572794.231228:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572794.231231:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616302464:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788572794.231253:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616302592:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572794.231279:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572794.231292:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572794/real 1788572794] req@ffff9ada4557c000 x1875449616302592/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572819 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572794.231307:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c000 x1875449616302592/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572819 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572794.231321:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572794.231323:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572794.231341:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616302592:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572794.231355:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616302720:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572794.231379:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572794.231391:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572794/real 1788572794] req@ffff9ada4557f480 x1875449616302720/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572819 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572794.231405:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616302720/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572819 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572794.231413:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572794.231416:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572794.231418:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616302720:192.168.202.113@tcp:38:kworker.0 00010000:00010000:1.0:1788572794.743058:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572795.767052:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572796.791080:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572797.815052:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572798.838995:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788572799.350920:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572799.350927:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788572799.350930:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572799.350938:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4711 00000100:00080000:2.0:1788572799.350946:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572799.350952:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788572799.350981:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616302848/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788572799.350999:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572799.351002:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788572799.351003:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572799.351005:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4711 00000100:00080000:2.0:1788572799.351007:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572799.351009:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572799.351015:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788572799.351022:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616302976/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788572799.351028:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572799.351030:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788572799.351033:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572799.351040:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4711 00000100:00100000:1.0:1788572799.351042:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616302848:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788572799.351043:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572799.351051:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572799.351055:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788572799.351063:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616303104/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788572799.351096:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572799.351123:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572799/real 1788572799] req@ffff9ada60625500 x1875449616302848/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572824 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572799.351136:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616302848/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572824 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572799.351144:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572799.351147:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572799.351149:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616302848:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788572799.351154:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616302976:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572799.351166:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572799.351173:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572799/real 1788572799] req@ffff9ada60624380 x1875449616302976/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572824 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572799.351180:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616302976/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572824 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572799.351184:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572799.351186:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572799.351196:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616302976:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572799.351215:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616303104:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572799.351227:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572799.351235:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572799/real 1788572799] req@ffff9ada60626a00 x1875449616303104/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572824 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572799.351249:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616303104/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572824 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572799.351256:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572799.351260:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572799.351263:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616303104:192.168.202.113@tcp:38:kworker.0 00010000:00010000:0.0:1788572799.862996:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572800.887049:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572801.911032:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572802.935028:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572803.959024:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788572804.470929:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572804.470937:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788572804.470940:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572804.470947:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4717 00000100:00080000:3.0:1788572804.470956:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572804.470961:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788572804.470991:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2300 x1875449616303232/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572804.471011:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572804.471014:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788572804.471015:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572804.471018:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4717 00000100:00080000:3.0:1788572804.471020:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572804.471022:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572804.471029:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572804.471036:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3800 x1875449616303360/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572804.471044:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572804.471046:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788572804.471047:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572804.471051:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4717 00000100:00100000:1.0:1788572804.471052:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616303232:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788572804.471053:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572804.471059:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572804.471064:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572804.471070:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1180 x1875449616303488/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788572804.471112:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572804.471145:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572804/real 1788572804] req@ffff9ada604e2300 x1875449616303232/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572829 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572804.471160:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2300 x1875449616303232/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572829 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572804.471168:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572804.471171:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572804.471173:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616303232:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788572804.471190:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616303360:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572804.471206:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572804.471214:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572804/real 1788572804] req@ffff9ada604e3800 x1875449616303360/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572829 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572804.471221:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3800 x1875449616303360/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572829 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572804.471225:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572804.471227:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572804.471242:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616303360:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572804.471246:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616303488:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572804.471259:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572804.471268:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572804/real 1788572804] req@ffff9ada604e1180 x1875449616303488/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572829 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572804.471279:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1180 x1875449616303488/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572829 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572804.471284:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572804.471286:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572804.471287:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616303488:192.168.202.113@tcp:38:kworker.0 00010000:00010000:3.0:1788572804.983003:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572806.007011:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572807.031016:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572808.055020:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572809.079070:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788572809.590990:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572809.591000:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788572809.591004:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572809.591012:0:77466:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4722 00000100:00080000:0.0:1788572809.591022:0:77466:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572809.591028:0:77466:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788572809.591063:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616303616/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788572809.591087:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572809.591090:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788572809.591091:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572809.591094:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4722 00000100:00080000:0.0:1788572809.591096:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572809.591098:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788572809.591106:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788572809.591114:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c380 x1875449616303744/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788572809.591123:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572809.591127:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788572809.591128:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572809.591133:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4722 00000100:00080000:0.0:1788572809.591137:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572809.591140:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788572809.591146:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616303616:192.168.202.113@tcp:250:kworker.0 00000008:00100000:0.0:1788572809.591149:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788572809.591165:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ce00 x1875449616303872/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788572809.591206:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572809.591237:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572809/real 1788572809] req@ffff9ada4557c700 x1875449616303616/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572834 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572809.591251:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616303616/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572834 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572809.591259:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572809.591262:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572809.591265:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616303616:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788572809.591270:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616303744:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572809.591285:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572809.591293:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572809/real 1788572809] req@ffff9ada4557c380 x1875449616303744/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572834 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572809.591301:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c380 x1875449616303744/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572834 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572809.591310:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572809.591312:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572809.591324:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616303744:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572809.591343:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616303872:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572809.591357:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572809.591367:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572809/real 1788572809] req@ffff9ada4557ce00 x1875449616303872/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572834 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572809.591377:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ce00 x1875449616303872/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572834 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572809.591383:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572809.591385:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572809.591387:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616303872:192.168.202.113@tcp:38:kworker.0 00010000:00010000:0.0:1788572810.103048:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788572810.230948:0:92554:0:(recover.c:348:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery finished DISCONN, rc = -110 00000080:00200008:3.0:1788572810.234679:0:92558:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1788572810.234688:0:92558:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:3.0:1788572810.234707:0:92558:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1788572810.234725:0:92558:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 64 lru_type: 1 00010000:00010000:3.0:1788572810.234742:0:92558:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 64 lru_type: 1 00000080:00200000:3.0:1788572810.234752:0:92558:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1788572810.234760:0:92558:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:3.0:1788572810.234765:0:92558:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:3.0:1788572810.234787:0:92558:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1ecfcb8 00010000:00010000:3.0:1788572810.234790:0:92558:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 64 lru_type: 1 00010000:00010000:3.0:1788572810.234798:0:92558:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 65 lru_type: 1 00000080:00200000:3.0:1788572810.234821:0:92558:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:3.0:1788572810.234827:0:92558:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1788572810.234832:0:92558:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 65 lru_type: 1 00010000:00010000:3.0:1788572810.234839:0:92558:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 65 lru_type: 1 00000080:00200000:3.0:1788572810.234843:0:92558:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1788572810.234846:0:92558:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:3.0:1788572810.234848:0:92558:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:3.0:1788572810.234869:0:92558:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1ecfbb0 00010000:00010000:3.0:1788572810.234872:0:92558:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 65 lru_type: 1 00010000:00010000:3.0:1788572810.234878:0:92558:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 66 lru_type: 1 00000080:00200000:3.0:1788572810.234885:0:92558:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:3.0:1788572810.234891:0:92558:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100400 00000080:00200008:3.0:1788572810.234903:0:92558:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200008:3.0:1788572810.235523:0:92558:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1788572810.235527:0:92558:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:3.0:1788572810.235534:0:92558:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1788572810.235541:0:92558:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 66 lru_type: 1 00010000:00010000:3.0:1788572810.235547:0:92558:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 66 lru_type: 1 00000080:00200000:3.0:1788572810.235552:0:92558:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1788572810.235555:0:92558:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:3.0:1788572810.235557:0:92558:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:3.0:1788572810.235561:0:92558:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1ecfcb8 00010000:00010000:3.0:1788572810.235563:0:92558:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 66 lru_type: 1 00010000:00010000:3.0:1788572810.235568:0:92558:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 67 lru_type: 1 00000080:00200008:3.0:1788572810.235584:0:92558:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1788572810.235587:0:92558:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:3.0:1788572810.235590:0:92558:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1788572810.235594:0:92558:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 67 lru_type: 1 00010000:00010000:3.0:1788572810.235600:0:92558:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 67 lru_type: 1 00000080:00200000:3.0:1788572810.235604:0:92558:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1788572810.235607:0:92558:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:3.0:1788572810.235608:0:92558:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:3.0:1788572810.235610:0:92558:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1ecfcb8 00010000:00010000:3.0:1788572810.235612:0:92558:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 67 lru_type: 1 00010000:00010000:3.0:1788572810.235617:0:92558:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 68 lru_type: 1 00000080:00200000:3.0:1788572810.235630:0:92558:0:(llite_lib.c:2903:ll_statfs()) VFS Op:sb=lustre (ffff9ada58b0a000) / proj=0 00000080:00000004:3.0:1788572810.235634:0:92558:0:(obd_class.h:1090:obd_statfs()) lustre-clilmv-ffff9ada58b0a000: age 4179, max_age 4726 00800000:00000004:3.0:1788572810.235641:0:92558:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: age 4488, max_age 4726 00000100:00100000:3.0:1788572810.235671:0:92558:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6059b9c0 going to sleep for 0 seconds 00000100:00100000:3.0:1788572810.235679:0:92558:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job df:35eb2d60-2de0-4b00-9815-d5153edd0057:92558:1875449616304000:192.168.202.113@tcp:41:df.0 00800000:00020000:3.0:1788572810.235691:0:92558:0:(lmv_obd.c:1471:lmv_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: can't stat MDS #0: rc = -110 00800000:00000004:3.0:1788572810.240168:0:92558:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: age 4488, max_age 4726 00000100:00100000:3.0:1788572810.240186:0:92558:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada6059b9c0 going to sleep for 0 seconds 00000100:00100000:3.0:1788572810.240192:0:92558:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job df:35eb2d60-2de0-4b00-9815-d5153edd0057:92558:1875449616304128:192.168.202.113@tcp:41:df.0 00800000:00020000:3.0:1788572810.240201:0:92558:0:(lmv_obd.c:1471:lmv_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: can't stat MDS #1: rc = -110 00010000:00010000:0.0:1788572811.127014:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572812.151033:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572813.175000:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572814.199026:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788572814.710977:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572814.710985:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788572814.710989:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572814.710998:0:77466:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4727 00000100:00080000:0.0:1788572814.711007:0:77466:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572814.711013:0:77466:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788572814.711052:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ce00 x1875449616304256/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788572814.711085:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572814.711089:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788572814.711091:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572814.711096:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4727 00000100:00080000:0.0:1788572814.711100:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572814.711104:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788572814.711114:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788572814.711127:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c380 x1875449616304384/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788572814.711139:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616304256:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788572814.711144:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572814.711153:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788572814.711154:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572814.711158:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4727 00000100:00080000:0.0:1788572814.711161:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572814.711164:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788572814.711168:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788572814.711177:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616304512/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788572814.711196:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572814.711234:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572814/real 1788572814] req@ffff9ada4557ce00 x1875449616304256/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572839 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572814.711256:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ce00 x1875449616304256/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572839 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572814.711269:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572814.711275:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572814.711282:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616304256:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788572814.711308:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616304384:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572814.711327:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572814.711337:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572814/real 1788572814] req@ffff9ada4557c380 x1875449616304384/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572839 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572814.711346:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c380 x1875449616304384/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572839 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572814.711351:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572814.711354:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572814.711356:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616304384:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572814.711360:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616304512:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572814.711371:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572814.711379:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572814/real 1788572814] req@ffff9ada4557c700 x1875449616304512/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572839 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572814.711404:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616304512/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572839 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572814.711413:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572814.711417:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572814.711420:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616304512:192.168.202.113@tcp:38:kworker.0 00010000:00010000:3.0:1788572815.223062:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572816.246990:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:2.0:1788572816.447798:0:92577:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=40086685 len=0 uarg=00007ffd4dbedb90 00000002:00000080:2.0:1788572816.447813:0:92577:0:(mdc_request.c:2317:mdc_iocontrol()) lustre-MDT0000-mdc-ffff9ada58b0a000: cmd=40086685 len=576 karg=ffff9ada45669000 uarg=0000000000000000 00000100:00080000:2.0:1788572816.447820:0:92577:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff9ada51140000 already not connected (conn 403, was 0): DISCONN 00000100:00080000:2.0:1788572816.447825:0:92577:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572816.447833:0:92577:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4732 00000100:00080000:2.0:1788572816.447842:0:92577:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572816.447848:0:92577:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572816.447899:0:92577:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788572816.447926:0:92577:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616304640/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788572816.447944:0:92577:0:(recover.c:337:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery started, waiting 20 seconds 00000100:00100000:1.0:1788572816.447970:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616304640:192.168.202.113@tcp:38:lctl.0 00000800:00000100:1.0:1788572816.448034:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572816.448060:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572816/real 1788572816] req@ffff9ada60624a80 x1875449616304640/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572841 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'lctl.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572816.448074:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616304640/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572841 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'lctl.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572816.448082:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572816.448085:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572816.448097:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616304640:192.168.202.113@tcp:38:lctl.0 00010000:00010000:0.0:1788572817.271023:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572818.295107:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572819.319088:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788572819.831018:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572819.831030:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788572819.831034:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572819.831044:0:77466:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4732 00000100:00080000:1.0:1788572819.831055:0:77466:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572819.831063:0:77466:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788572819.831102:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c700 x1875449616304768/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572819.831128:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572819.831131:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788572819.831132:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572819.831135:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4734 00000100:00080000:1.0:1788572819.831138:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572819.831141:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572819.831150:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572819.831161:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c380 x1875449616304896/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572819.831170:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572819.831172:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788572819.831174:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572819.831179:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4732 00000100:00100000:3.0:1788572819.831179:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616304768:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788572819.831184:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572819.831189:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572819.831192:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572819.831199:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f100 x1875449616305024/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788572819.831253:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572819.831289:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572819/real 1788572819] req@ffff9ada4566c700 x1875449616304768/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572844 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572819.831309:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c700 x1875449616304768/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572844 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572819.831320:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572819.831324:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572819.831328:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616304768:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572819.831351:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616304896:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572819.831374:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572819.831386:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572819/real 1788572819] req@ffff9ada4566c380 x1875449616304896/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572844 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572819.831398:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c380 x1875449616304896/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572844 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572819.831405:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572819.831409:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572819.831423:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616304896:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572819.831430:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616305024:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572819.831451:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572819.831466:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572819/real 1788572819] req@ffff9ada4566f100 x1875449616305024/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572844 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572819.831484:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f100 x1875449616305024/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572844 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572819.831492:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572819.831496:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572819.831499:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616305024:192.168.202.113@tcp:38:kworker.0 00010000:00010000:0.0:1788572820.343012:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572821.367046:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572822.391021:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572823.415013:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572824.439044:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788572824.951001:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572824.951009:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788572824.951012:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572824.951021:0:77466:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4737 00000100:00080000:2.0:1788572824.951048:0:77466:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572824.951056:0:77466:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788572824.951090:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616305152/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788572824.951111:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572824.951114:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788572824.951115:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572824.951118:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4737 00000100:00080000:2.0:1788572824.951120:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572824.951122:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572824.951129:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788572824.951138:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616305280/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788572824.951146:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572824.951149:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788572824.951150:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572824.951153:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4737 00000100:00080000:2.0:1788572824.951155:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1788572824.951158:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616305152:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788572824.951159:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572824.951166:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788572824.951173:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616305408/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788572824.951225:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572824.951254:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572824/real 1788572824] req@ffff9ada60625880 x1875449616305152/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572849 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572824.951268:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616305152/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572849 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572824.951276:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572824.951279:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572824.951281:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616305152:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572824.951287:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616305280:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572824.951301:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572824.951309:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572824/real 1788572824] req@ffff9ada60625c00 x1875449616305280/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572849 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572824.951318:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616305280/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572849 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572824.951323:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572824.951326:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572824.951336:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616305280:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572824.951356:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616305408:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572824.951370:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572824.951382:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572824/real 1788572824] req@ffff9ada60625f80 x1875449616305408/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572849 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572824.951393:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616305408/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572849 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572824.951398:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572824.951400:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572824.951402:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616305408:192.168.202.113@tcp:38:kworker.0 00010000:00010000:1.0:1788572825.462988:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572826.487049:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572827.510995:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572828.535051:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572829.559099:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788572830.070976:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572830.070988:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788572830.070993:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572830.071004:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4742 00000100:00080000:2.0:1788572830.071016:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572830.071024:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788572830.071071:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616305536/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788572830.071098:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572830.071102:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788572830.071103:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572830.071106:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4742 00000100:00080000:2.0:1788572830.071109:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572830.071111:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572830.071120:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788572830.071147:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616305664/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788572830.071156:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572830.071159:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788572830.071161:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572830.071164:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4742 00000100:00080000:2.0:1788572830.071167:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572830.071170:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572830.071177:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572830.071184:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616305536:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788572830.071190:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616305792/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788572830.071277:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572830.071318:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572830/real 1788572830] req@ffff9ada60627480 x1875449616305536/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572855 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572830.071335:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616305536/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572855 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572830.071344:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572830.071348:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572830.071351:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616305536:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572830.071357:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616305664:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572830.071376:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572830.071385:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572830/real 1788572830] req@ffff9ada60625180 x1875449616305664/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572855 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572830.071393:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616305664/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572855 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572830.071398:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572830.071400:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572830.071414:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616305664:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572830.071438:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616305792:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572830.071458:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572830.071466:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572830/real 1788572830] req@ffff9ada60624e00 x1875449616305792/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572855 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572830.071475:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616305792/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572855 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572830.071480:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572830.071482:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572830.071484:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616305792:192.168.202.113@tcp:38:kworker.0 00010000:00010000:3.0:1788572830.583063:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572831.607071:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572832.631051:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572833.655157:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572834.679505:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788572835.191022:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572835.191037:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788572835.191042:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572835.191054:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4747 00000100:00080000:2.0:1788572835.191067:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572835.191076:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788572835.191118:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616305920/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788572835.191142:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572835.191146:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788572835.191147:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572835.191151:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4747 00000100:00080000:2.0:1788572835.191154:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572835.191157:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572835.191168:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572835.191176:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616305920:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788572835.191179:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616306048/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788572835.191189:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572835.191193:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788572835.191194:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572835.191197:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4747 00000100:00080000:2.0:1788572835.191199:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572835.191201:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572835.191206:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788572835.191215:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626300 x1875449616306176/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788572835.191277:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572835.191316:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572835/real 1788572835] req@ffff9ada60626d80 x1875449616305920/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572860 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572835.191335:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616305920/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572860 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572835.191345:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572835.191349:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572835.191352:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616305920:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572835.191376:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616306048:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572835.191393:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572835.191401:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572835/real 1788572835] req@ffff9ada60627b80 x1875449616306048/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572860 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572835.191410:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616306048/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572860 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572835.191415:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572835.191417:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572835.191429:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616306048:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572835.191433:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616306176:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572835.191447:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572835.191457:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572835/real 1788572835] req@ffff9ada60626300 x1875449616306176/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572860 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572835.191467:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626300 x1875449616306176/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572860 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572835.191474:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572835.191476:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572835.191478:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616306176:192.168.202.113@tcp:38:kworker.0 00010000:00010000:0.0:1788572835.703106:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788572836.471006:0:92577:0:(recover.c:348:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery finished DISCONN, rc = -110 00000080:00200008:2.0:1788572836.475676:0:92581:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1788572836.475689:0:92581:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788572836.475719:0:92581:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788572836.475743:0:92581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 68 lru_type: 1 00010000:00010000:2.0:1788572836.475759:0:92581:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 68 lru_type: 1 00000080:00200000:2.0:1788572836.475767:0:92581:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788572836.475775:0:92581:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788572836.475779:0:92581:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:2.0:1788572836.475788:0:92581:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1e7fcb8 00010000:00010000:2.0:1788572836.475791:0:92581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 68 lru_type: 1 00010000:00010000:2.0:1788572836.475800:0:92581:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 69 lru_type: 1 00000080:00200000:2.0:1788572836.475826:0:92581:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788572836.475830:0:92581:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788572836.475834:0:92581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 69 lru_type: 1 00010000:00010000:2.0:1788572836.475845:0:92581:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 69 lru_type: 1 00000080:00200000:2.0:1788572836.475849:0:92581:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788572836.475852:0:92581:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788572836.475853:0:92581:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:2.0:1788572836.475855:0:92581:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1e7fbb0 00010000:00010000:2.0:1788572836.475858:0:92581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 69 lru_type: 1 00010000:00010000:2.0:1788572836.475891:0:92581:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 70 lru_type: 1 00000080:00200000:2.0:1788572836.475903:0:92581:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:2.0:1788572836.475913:0:92581:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100400 00000080:00200008:2.0:1788572836.475932:0:92581:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200008:2.0:1788572836.476639:0:92581:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1788572836.476643:0:92581:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788572836.476650:0:92581:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788572836.476656:0:92581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 70 lru_type: 1 00010000:00010000:2.0:1788572836.476663:0:92581:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 70 lru_type: 1 00000080:00200000:2.0:1788572836.476669:0:92581:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788572836.476672:0:92581:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788572836.476675:0:92581:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:2.0:1788572836.476678:0:92581:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1e7fcb8 00010000:00010000:2.0:1788572836.476681:0:92581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 70 lru_type: 1 00010000:00010000:2.0:1788572836.476686:0:92581:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 71 lru_type: 1 00000080:00200008:2.0:1788572836.476703:0:92581:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1788572836.476706:0:92581:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788572836.476710:0:92581:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788572836.476713:0:92581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 71 lru_type: 1 00010000:00010000:2.0:1788572836.476719:0:92581:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 71 lru_type: 1 00000080:00200000:2.0:1788572836.476724:0:92581:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788572836.476727:0:92581:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788572836.476729:0:92581:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:2.0:1788572836.476731:0:92581:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1e7fcb8 00010000:00010000:2.0:1788572836.476733:0:92581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 71 lru_type: 1 00010000:00010000:2.0:1788572836.476737:0:92581:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 72 lru_type: 1 00000080:00200000:2.0:1788572836.476753:0:92581:0:(llite_lib.c:2903:ll_statfs()) VFS Op:sb=lustre (ffff9ada58b0a000) / proj=0 00000080:00000004:2.0:1788572836.476758:0:92581:0:(obd_class.h:1090:obd_statfs()) lustre-clilmv-ffff9ada58b0a000: age 4179, max_age 4753 00800000:00000004:2.0:1788572836.476770:0:92581:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: age 4488, max_age 4753 00000100:00100000:2.0:1788572836.476814:0:92581:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60703300 going to sleep for 0 seconds 00000100:00100000:2.0:1788572836.476827:0:92581:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job df:35eb2d60-2de0-4b00-9815-d5153edd0057:92581:1875449616306304:192.168.202.113@tcp:41:df.0 00800000:00020000:2.0:1788572836.476842:0:92581:0:(lmv_obd.c:1471:lmv_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: can't stat MDS #0: rc = -110 00800000:00000004:2.0:1788572836.476846:0:92581:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: age 4488, max_age 4753 00000100:00100000:2.0:1788572836.476853:0:92581:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60703300 going to sleep for 0 seconds 00000100:00100000:2.0:1788572836.476856:0:92581:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job df:35eb2d60-2de0-4b00-9815-d5153edd0057:92581:1875449616306432:192.168.202.113@tcp:41:df.0 00800000:00020000:2.0:1788572836.476862:0:92581:0:(lmv_obd.c:1471:lmv_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: can't stat MDS #1: rc = -110 00010000:00010000:3.0:1788572836.727062:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572837.751135:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572838.775023:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572839.799117:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788572840.311027:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572840.311035:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788572840.311039:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572840.311047:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4752 00000100:00080000:2.0:1788572840.311056:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572840.311062:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788572840.311096:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627800 x1875449616306560/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788572840.311120:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572840.311123:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788572840.311124:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572840.311127:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4752 00000100:00080000:2.0:1788572840.311129:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572840.311131:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572840.311139:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788572840.311150:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626300 x1875449616306688/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788572840.311160:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572840.311163:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788572840.311164:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572840.311170:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4752 00000100:00100000:3.0:1788572840.311171:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616306560:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788572840.311172:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572840.311179:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572840.311182:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788572840.311191:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616306816/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788572840.311232:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572840.311261:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572840/real 1788572840] req@ffff9ada60627800 x1875449616306560/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572865 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572840.311276:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627800 x1875449616306560/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572865 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572840.311284:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572840.311287:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572840.311290:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616306560:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572840.311310:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616306688:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572840.311348:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572840.311359:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572840/real 1788572840] req@ffff9ada60626300 x1875449616306688/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572865 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572840.311368:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626300 x1875449616306688/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572865 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572840.311374:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572840.311376:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572840.311379:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616306688:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572840.311382:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616306816:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572840.311395:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572840.311401:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572840/real 1788572840] req@ffff9ada60627b80 x1875449616306816/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572865 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572840.311408:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616306816/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572865 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572840.311413:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572840.311415:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572840.311417:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616306816:192.168.202.113@tcp:38:kworker.0 00010000:00010000:1.0:1788572840.823014:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572841.847050:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:2.0:1788572842.688597:0:92600:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=40086685 len=0 uarg=00007ffcb8b6f240 00000002:00000080:2.0:1788572842.688613:0:92600:0:(mdc_request.c:2317:mdc_iocontrol()) lustre-MDT0000-mdc-ffff9ada58b0a000: cmd=40086685 len=576 karg=ffff9ada45669000 uarg=0000000000000000 00000100:00080000:2.0:1788572842.688619:0:92600:0:(import.c:214:ptlrpc_set_import_discon()) mdc: import ffff9ada51140000 already not connected (conn 409, was 0): DISCONN 00000100:00080000:2.0:1788572842.688624:0:92600:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572842.688631:0:92600:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4757 00000100:00080000:2.0:1788572842.688639:0:92600:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572842.688645:0:92600:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572842.688655:0:92600:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788572842.688678:0:92600:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616306944/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788572842.688699:0:92600:0:(recover.c:337:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery started, waiting 20 seconds 00000100:00100000:3.0:1788572842.688737:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616306944:192.168.202.113@tcp:38:lctl.0 00000800:00000100:3.0:1788572842.688791:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572842.688813:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572842/real 1788572842] req@ffff9ada60626d80 x1875449616306944/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572867 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'lctl.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572842.688829:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616306944/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572867 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'lctl.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572842.688837:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572842.688840:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572842.689056:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616306944:192.168.202.113@tcp:38:lctl.0 00010000:00010000:3.0:1788572842.871031:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572843.895036:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572844.919012:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788572845.430952:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572845.430962:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788572845.430966:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572845.430977:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4757 00000100:00080000:0.0:1788572845.430988:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572845.430995:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788572845.431031:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616307072/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788572845.431059:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572845.431063:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788572845.431065:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572845.431068:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4760 00000100:00080000:0.0:1788572845.431072:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572845.431076:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788572845.431085:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788572845.431096:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c000 x1875449616307200/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788572845.431106:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572845.431110:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788572845.431112:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572845.431116:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4757 00000100:00080000:0.0:1788572845.431119:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1788572845.431125:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616307072:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788572845.431133:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788572845.431140:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788572845.431152:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616307328/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788572845.431191:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572845.431226:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572845/real 1788572845] req@ffff9ada4557f480 x1875449616307072/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572870 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572845.431246:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616307072/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572870 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572845.431258:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572845.431262:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572845.431266:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616307072:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572845.431274:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616307200:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572845.431299:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572845.431313:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572845/real 1788572845] req@ffff9ada4557c000 x1875449616307200/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572870 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572845.431327:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c000 x1875449616307200/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572870 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572845.431335:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572845.431339:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572845.431355:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616307200:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572845.431392:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616307328:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572845.431416:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572845.431428:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572845/real 1788572845] req@ffff9ada4557ca80 x1875449616307328/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572870 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572845.431476:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616307328/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572870 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572845.431485:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572845.431488:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572845.431492:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616307328:192.168.202.113@tcp:38:kworker.0 00010000:00010000:3.0:1788572845.943013:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572846.966989:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572847.991012:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572849.014992:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572850.038992:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788572850.550990:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572850.550998:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788572850.551003:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572850.551012:0:91222:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4763 00000100:00080000:2.0:1788572850.551021:0:91222:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572850.551028:0:91222:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788572850.551061:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616307456/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788572850.551081:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572850.551083:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788572850.551085:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572850.551087:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4763 00000100:00080000:2.0:1788572850.551089:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572850.551092:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572850.551101:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788572850.551115:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616307584/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788572850.551128:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572850.551132:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788572850.551137:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1788572850.551137:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616307456:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788572850.551147:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4763 00000100:00080000:2.0:1788572850.551151:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572850.551154:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572850.551159:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788572850.551169:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616307712/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788572850.551227:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616307584:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572850.551263:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616307712:192.168.202.113@tcp:38:kworker.0 00010000:00010000:3.0:1788572851.063069:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572852.087022:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572853.111053:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1788572853.623138:0:2370:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:2.0:1788572853.623183:0:2370:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1788572853.623200:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788572853.623244:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788572853.623254:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:3.0:1788572853.623305:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572850/real 1788572853] req@ffff9ada60625180 x1875449616307584/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572875 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572853.623334:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616307584/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572875 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572853.623347:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572853.623368:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572853.623384:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616307584:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572853.623394:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572850/real 1788572853] req@ffff9ada60627480 x1875449616307712/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572875 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572853.623410:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616307712/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572875 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572853.623420:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572853.623431:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572853.623435:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616307712:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572853.623447:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572850/real 1788572853] req@ffff9ada60624e00 x1875449616307456/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572875 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572853.623460:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616307456/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572875 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572853.623468:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572853.623472:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572853.623475:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616307456:192.168.202.113@tcp:250:kworker.0 00010000:00010000:3.0:1788572854.134972:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572855.159076:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788572855.670985:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572855.670996:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788572855.671001:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572855.671013:0:91222:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4768 00000100:00080000:3.0:1788572855.671025:0:91222:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572855.671033:0:91222:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788572855.671075:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1880 x1875449616307840/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572855.671106:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572855.671110:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788572855.671111:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572855.671120:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4768 00000100:00080000:3.0:1788572855.671124:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572855.671129:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572855.671145:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572855.671163:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0380 x1875449616307968/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788572855.671165:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616307840:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788572855.671177:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572855.671183:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788572855.671188:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572855.671194:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4768 00000100:00080000:3.0:1788572855.671198:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572855.671203:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572855.671208:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572855.671222:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0700 x1875449616308096/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788572855.671276:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572855.671323:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572855/real 1788572855] req@ffff9ada604e1880 x1875449616307840/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572880 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572855.671346:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1880 x1875449616307840/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572880 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572855.671357:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572855.671362:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572855.671366:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616307840:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788572855.671399:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616307968:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572855.671426:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572855.671440:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572855/real 1788572855] req@ffff9ada604e0380 x1875449616307968/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572880 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572855.671453:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0380 x1875449616307968/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572880 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572855.671461:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572855.671464:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572855.671482:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616307968:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572855.671489:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616308096:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572855.671515:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572855.671532:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572855/real 1788572855] req@ffff9ada604e0700 x1875449616308096/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572880 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572855.671547:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0700 x1875449616308096/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572880 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572855.671563:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572855.671567:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572855.671573:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616308096:192.168.202.113@tcp:38:kworker.0 00010000:00010000:1.0:1788572856.182980:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572857.207099:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572858.231052:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572859.255025:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572860.279060:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788572860.790940:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572860.790948:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788572860.790952:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572860.790961:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4773 00000100:00080000:3.0:1788572860.790970:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572860.790977:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788572860.791011:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3480 x1875449616308224/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572860.791034:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572860.791036:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788572860.791038:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572860.791041:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4773 00000100:00080000:3.0:1788572860.791043:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572860.791045:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572860.791053:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572860.791063:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0700 x1875449616308352/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572860.791070:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572860.791073:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788572860.791074:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572860.791076:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4773 00000100:00080000:3.0:1788572860.791078:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572860.791081:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572860.791085:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572860.791089:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616308224:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572860.791097:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0380 x1875449616308480/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788572860.791146:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572860.791177:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572860/real 1788572860] req@ffff9ada604e3480 x1875449616308224/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572885 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572860.791191:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3480 x1875449616308224/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572885 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572860.791199:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572860.791202:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572860.791205:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616308224:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788572860.791210:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616308352:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572860.791224:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572860.791232:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572860/real 1788572860] req@ffff9ada604e0700 x1875449616308352/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572885 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572860.791240:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0700 x1875449616308352/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572885 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572860.791250:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572860.791252:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572860.791265:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616308352:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572860.791293:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616308480:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572860.791315:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572860.791332:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572860/real 1788572860] req@ffff9ada604e0380 x1875449616308480/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572885 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572860.791340:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0380 x1875449616308480/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572885 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572860.791346:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572860.791348:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572860.791350:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616308480:192.168.202.113@tcp:38:kworker.0 00010000:00010000:3.0:1788572861.303002:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572862.327044:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788572862.711005:0:92600:0:(recover.c:348:ptlrpc_recover_import()) lustre-MDT0000_UUID: recovery finished DISCONN, rc = -110 00000080:00200008:2.0:1788572862.714477:0:92603:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1788572862.714489:0:92603:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788572862.714512:0:92603:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788572862.714533:0:92603:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 72 lru_type: 1 00010000:00010000:2.0:1788572862.714561:0:92603:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 72 lru_type: 1 00000080:00200000:2.0:1788572862.714570:0:92603:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788572862.714578:0:92603:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788572862.714583:0:92603:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:2.0:1788572862.714591:0:92603:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1f77cb8 00010000:00010000:2.0:1788572862.714596:0:92603:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 72 lru_type: 1 00010000:00010000:2.0:1788572862.714605:0:92603:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 73 lru_type: 1 00000080:00200000:2.0:1788572862.714637:0:92603:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788572862.714647:0:92603:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788572862.714654:0:92603:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 73 lru_type: 1 00010000:00010000:2.0:1788572862.714662:0:92603:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 73 lru_type: 1 00000080:00200000:2.0:1788572862.714668:0:92603:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788572862.714672:0:92603:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788572862.714675:0:92603:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:2.0:1788572862.714679:0:92603:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1f77bb0 00010000:00010000:2.0:1788572862.714681:0:92603:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 73 lru_type: 1 00010000:00010000:2.0:1788572862.714689:0:92603:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 74 lru_type: 1 00000080:00200000:2.0:1788572862.714697:0:92603:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:2.0:1788572862.714705:0:92603:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100400 00000080:00200008:2.0:1788572862.714722:0:92603:0:(file.c:479:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200008:2.0:1788572862.715655:0:92603:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1788572862.715661:0:92603:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788572862.715671:0:92603:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788572862.715680:0:92603:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 74 lru_type: 1 00010000:00010000:2.0:1788572862.715688:0:92603:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 74 lru_type: 1 00000080:00200000:2.0:1788572862.715698:0:92603:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788572862.715701:0:92603:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788572862.715703:0:92603:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:2.0:1788572862.715707:0:92603:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1f77cb8 00010000:00010000:2.0:1788572862.715709:0:92603:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 74 lru_type: 1 00010000:00010000:2.0:1788572862.715715:0:92603:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 75 lru_type: 1 00000080:00200008:2.0:1788572862.715733:0:92603:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1788572862.715736:0:92603:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:2.0:1788572862.715739:0:92603:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1788572862.715743:0:92603:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 75 lru_type: 1 00010000:00010000:2.0:1788572862.715749:0:92603:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 75 lru_type: 1 00000080:00200000:2.0:1788572862.715755:0:92603:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1788572862.715759:0:92603:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:2.0:1788572862.715761:0:92603:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:2.0:1788572862.715765:0:92603:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec1f77cb8 00010000:00010000:2.0:1788572862.715768:0:92603:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 75 lru_type: 1 00010000:00010000:2.0:1788572862.715809:0:92603:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 76 lru_type: 1 00000080:00200000:2.0:1788572862.715832:0:92603:0:(llite_lib.c:2903:ll_statfs()) VFS Op:sb=lustre (ffff9ada58b0a000) / proj=0 00000080:00000004:2.0:1788572862.715836:0:92603:0:(obd_class.h:1090:obd_statfs()) lustre-clilmv-ffff9ada58b0a000: age 4179, max_age 4779 00800000:00000004:2.0:1788572862.715848:0:92603:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: age 4488, max_age 4779 00000100:00100000:2.0:1788572862.715916:0:92603:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60703300 going to sleep for 0 seconds 00000100:00100000:2.0:1788572862.715927:0:92603:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job df:35eb2d60-2de0-4b00-9815-d5153edd0057:92603:1875449616308608:192.168.202.113@tcp:41:df.0 00800000:00020000:2.0:1788572862.715942:0:92603:0:(lmv_obd.c:1471:lmv_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: can't stat MDS #0: rc = -110 00800000:00000004:2.0:1788572862.715945:0:92603:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: age 4488, max_age 4779 00000100:00100000:2.0:1788572862.715952:0:92603:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada60703300 going to sleep for 0 seconds 00000100:00100000:2.0:1788572862.715954:0:92603:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job df:35eb2d60-2de0-4b00-9815-d5153edd0057:92603:1875449616308736:192.168.202.113@tcp:41:df.0 00800000:00020000:2.0:1788572862.715959:0:92603:0:(lmv_obd.c:1471:lmv_statfs()) lustre-MDT0001-mdc-ffff9ada58b0a000: can't stat MDS #1: rc = -110 00010000:00010000:1.0:1788572863.351127:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572864.375091:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572865.399045:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788572865.911011:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572865.911021:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788572865.911025:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572865.911037:0:77466:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4778 00000100:00080000:1.0:1788572865.911049:0:77466:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572865.911056:0:77466:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788572865.911102:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ca80 x1875449616308864/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572865.911132:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572865.911135:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788572865.911136:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572865.911139:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4778 00000100:00080000:1.0:1788572865.911141:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572865.911144:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572865.911153:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572865.911162:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d500 x1875449616308992/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572865.911170:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572865.911173:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788572865.911174:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572865.911177:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4778 00000100:00080000:1.0:1788572865.911179:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572865.911181:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572865.911184:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572865.911195:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d180 x1875449616309120/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788572865.911210:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616308864:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788572865.911279:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572865.911310:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572865/real 1788572865] req@ffff9ada4566ca80 x1875449616308864/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572890 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572865.911326:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ca80 x1875449616308864/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572890 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572865.911335:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572865.911338:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572865.911341:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616308864:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572865.911347:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616308992:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572865.911362:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572865.911370:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572865/real 1788572865] req@ffff9ada4566d500 x1875449616308992/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572890 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572865.911378:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d500 x1875449616308992/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572890 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572865.911383:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572865.911385:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572865.911387:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616308992:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572865.911408:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616309120:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572865.911420:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572865.911426:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572865/real 1788572865] req@ffff9ada4566d180 x1875449616309120/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572890 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572865.911434:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d180 x1875449616309120/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572890 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572865.911439:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572865.911441:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572865.911443:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616309120:192.168.202.113@tcp:38:kworker.0 00010000:00010000:3.0:1788572866.422973:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572867.447061:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572868.471049:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572869.495064:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572870.519147:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788572871.031032:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572871.031048:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788572871.031055:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572871.031069:0:77466:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4783 00000100:00080000:2.0:1788572871.031082:0:77466:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572871.031093:0:77466:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788572871.031143:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616309248/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788572871.031175:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572871.031179:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788572871.031181:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572871.031186:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4783 00000100:00080000:2.0:1788572871.031191:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572871.031196:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572871.031208:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572871.031222:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616309248:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788572871.031224:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616309376/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788572871.031239:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572871.031243:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788572871.031245:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572871.031250:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4783 00000100:00080000:2.0:1788572871.031253:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572871.031257:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572871.031262:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788572871.031274:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616309504/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788572871.031307:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572871.031345:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572871/real 1788572871] req@ffff9ada60625f80 x1875449616309248/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572896 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572871.031363:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616309248/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572896 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572871.031372:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572871.031375:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572871.031379:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616309248:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572871.031408:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616309376:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572871.031427:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572871.031438:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572871/real 1788572871] req@ffff9ada60625c00 x1875449616309376/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572896 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572871.031452:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616309376/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572896 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572871.031461:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572871.031465:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572871.031468:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616309376:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572871.031475:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616309504:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572871.031497:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572871.031506:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572871/real 1788572871] req@ffff9ada60625880 x1875449616309504/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572896 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572871.031514:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616309504/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572896 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572871.031520:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572871.031522:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572871.031524:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616309504:192.168.202.113@tcp:38:kworker.0 00010000:00010000:1.0:1788572871.543033:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572872.567065:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572873.591026:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572874.615057:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572875.639081:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788572876.150972:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572876.150983:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788572876.150988:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572876.150998:0:77466:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4788 00000100:00080000:2.0:1788572876.151009:0:77466:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572876.151018:0:77466:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788572876.151074:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616309632/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788572876.151108:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572876.151113:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788572876.151115:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572876.151119:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4788 00000100:00080000:2.0:1788572876.151124:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572876.151128:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572876.151139:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788572876.151154:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616309760/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788572876.151162:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616309632:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788572876.151172:0:77466:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572876.151177:0:77466:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788572876.151180:0:77466:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572876.151185:0:77466:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4788 00000100:00080000:2.0:1788572876.151190:0:77466:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572876.151195:0:77466:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572876.151201:0:77466:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788572876.151214:0:77466:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616309888/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788572876.151227:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572876.151264:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572876/real 1788572876] req@ffff9ada60624a80 x1875449616309632/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572901 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572876.151284:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616309632/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572901 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572876.151296:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572876.151301:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572876.151305:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616309632:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572876.151328:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616309760:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572876.151349:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572876.151360:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572876/real 1788572876] req@ffff9ada60626a00 x1875449616309760/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572901 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572876.151373:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616309760/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572901 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572876.151394:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572876.151397:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572876.151401:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616309760:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572876.151407:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616309888:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572876.151424:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572876.151434:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572876/real 1788572876] req@ffff9ada60624380 x1875449616309888/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572901 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572876.151446:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616309888/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572901 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572876.151456:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572876.151458:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572876.151461:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616309888:192.168.202.113@tcp:38:kworker.0 00010000:00010000:3.0:1788572876.663044:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572877.687024:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572878.711007:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572879.735062:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572880.759020:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788572881.271015:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572881.271025:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788572881.271030:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572881.271041:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4793 00000100:00080000:0.0:1788572881.271054:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572881.271063:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788572881.271114:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616310016/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788572881.271138:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572881.271142:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788572881.271144:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572881.271149:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4793 00000100:00080000:0.0:1788572881.271154:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572881.271159:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788572881.271168:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616310016:192.168.202.113@tcp:250:kworker.0 00000008:00100000:0.0:1788572881.271169:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788572881.271190:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616310144/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788572881.271203:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572881.271207:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788572881.271209:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572881.271214:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4793 00000100:00080000:0.0:1788572881.271218:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572881.271222:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000800:00000100:3.0:1788572881.271228:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000008:00100000:0.0:1788572881.271229:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788572881.271260:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616310272/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788572881.271268:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572881/real 1788572881] req@ffff9ada4557f800 x1875449616310016/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572906 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572881.271293:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616310016/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572906 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572881.271305:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572881.271310:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572881.271314:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616310016:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572881.271338:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616310144:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572881.271364:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572881.271378:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572881/real 1788572881] req@ffff9ada4557d880 x1875449616310144/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572906 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572881.271391:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616310144/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572906 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572881.271399:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572881.271402:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572881.271406:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616310144:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572881.271413:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616310272:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572881.271432:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572881.271444:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572881/real 1788572881] req@ffff9ada4557fb80 x1875449616310272/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572906 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572881.271457:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616310272/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572906 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572881.271462:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572881.271465:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572881.271467:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616310272:192.168.202.113@tcp:38:kworker.0 00010000:00010000:1.0:1788572881.783006:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572882.807006:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572883.830976:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572884.855021:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572885.878977:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788572886.390964:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572886.390972:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788572886.390976:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572886.390983:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4798 00000100:00080000:2.0:1788572886.390997:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572886.391003:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788572886.391037:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616310400/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788572886.391066:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572886.391069:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788572886.391071:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572886.391074:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4798 00000100:00080000:2.0:1788572886.391078:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572886.391082:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572886.391091:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572886.391103:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616310400:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788572886.391106:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616310528/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788572886.391125:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572886.391129:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788572886.391131:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572886.391134:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4798 00000100:00080000:2.0:1788572886.391163:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572886.391168:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572886.391172:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1788572886.391176:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788572886.391184:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627100 x1875449616310656/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788572886.391216:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572886/real 1788572886] req@ffff9ada60625500 x1875449616310400/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572911 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572886.391233:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616310400/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572911 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572886.391240:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572886.391242:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572886.391245:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616310400:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572886.391262:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616310528:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572886.391278:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572886.391286:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572886/real 1788572886] req@ffff9ada60626680 x1875449616310528/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572911 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572886.391292:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616310528/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572911 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572886.391297:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572886.391299:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572886.391308:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616310528:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572886.391312:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616310656:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572886.391322:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572886.391328:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572886/real 1788572886] req@ffff9ada60627100 x1875449616310656/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572911 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572886.391335:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627100 x1875449616310656/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572911 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572886.391339:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572886.391341:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572886.391343:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616310656:192.168.202.113@tcp:38:kworker.0 00010000:00010000:0.0:1788572886.903020:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572887.927030:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572888.951037:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572889.975045:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572890.999014:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788572891.510990:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572891.511003:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788572891.511008:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572891.511019:0:91222:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4804 00000100:00080000:1.0:1788572891.511032:0:91222:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572891.511041:0:91222:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788572891.511087:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f480 x1875449616310784/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572891.511118:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572891.511122:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788572891.511124:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572891.511128:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4804 00000100:00080000:1.0:1788572891.511132:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572891.511137:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572891.511148:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572891.511162:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e300 x1875449616310912/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788572891.511162:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616310784:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788572891.511176:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572891.511179:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788572891.511180:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572891.511183:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4804 00000100:00080000:1.0:1788572891.511186:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572891.511189:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572891.511192:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572891.511201:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ea00 x1875449616311040/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788572891.511247:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572891.511282:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572891/real 1788572891] req@ffff9ada4566f480 x1875449616310784/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572916 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572891.511298:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f480 x1875449616310784/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572916 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572891.511310:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572891.511314:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572891.511319:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616310784:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572891.511346:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616310912:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572891.511363:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572891.511371:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572891/real 1788572891] req@ffff9ada4566e300 x1875449616310912/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572916 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572891.511379:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e300 x1875449616310912/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572916 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572891.511384:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572891.511386:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572891.511388:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616310912:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572891.511392:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616311040:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572891.511408:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572891.511417:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572891/real 1788572891] req@ffff9ada4566ea00 x1875449616311040/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572916 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572891.511428:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ea00 x1875449616311040/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572916 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572891.511436:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572891.511440:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572891.511443:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616311040:192.168.202.113@tcp:38:kworker.0 00010000:00010000:2.0:1788572892.023094:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572893.047050:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572894.071029:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788572895.094962:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572896.118946:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788572896.630908:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572896.630918:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788572896.630922:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572896.630932:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4809 00000100:00080000:1.0:1788572896.630942:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572896.630949:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788572896.630992:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ed80 x1875449616311168/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572896.631021:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572896.631026:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788572896.631028:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572896.631032:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4809 00000100:00080000:1.0:1788572896.631036:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572896.631040:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572896.631052:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572896.631064:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f800 x1875449616311296/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572896.631072:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572896.631074:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788572896.631075:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572896.631079:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4809 00000100:00080000:1.0:1788572896.631082:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572896.631085:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572896.631090:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572896.631099:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616311168:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788572896.631100:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566dc00 x1875449616311424/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788572896.631172:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572896.631201:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572896/real 1788572896] req@ffff9ada4566ed80 x1875449616311168/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572921 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572896.631213:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ed80 x1875449616311168/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572921 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572896.631221:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572896.631224:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572896.631227:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616311168:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572896.631231:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616311296:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572896.631246:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572896.631257:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572896/real 1788572896] req@ffff9ada4566f800 x1875449616311296/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572921 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572896.631292:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f800 x1875449616311296/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572921 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572896.631301:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572896.631304:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572896.631306:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616311296:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572896.631325:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616311424:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572896.631339:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572896.631346:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572896/real 1788572896] req@ffff9ada4566dc00 x1875449616311424/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572921 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572896.631354:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566dc00 x1875449616311424/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572921 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572896.631359:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572896.631360:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572896.631362:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616311424:192.168.202.113@tcp:38:kworker.0 00010000:00010000:1.0:1788572897.143045:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572898.167056:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572899.191125:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572900.215053:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572901.239001:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788572901.750922:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572901.750930:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788572901.750934:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572901.750943:0:91222:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4814 00000100:00080000:1.0:1788572901.750951:0:91222:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572901.750958:0:91222:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788572901.750991:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566fb80 x1875449616311552/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572901.751021:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572901.751026:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788572901.751028:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572901.751033:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4814 00000100:00080000:1.0:1788572901.751037:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572901.751042:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572901.751051:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572901.751063:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c000 x1875449616311680/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788572901.751069:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616311552:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788572901.751075:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572901.751079:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788572901.751081:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572901.751084:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4814 00000100:00080000:1.0:1788572901.751087:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572901.751090:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572901.751093:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572901.751102:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566df80 x1875449616311808/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788572901.751126:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572901.751155:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572901/real 1788572901] req@ffff9ada4566fb80 x1875449616311552/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572926 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572901.751170:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566fb80 x1875449616311552/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572926 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572901.751178:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572901.751181:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572901.751184:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616311552:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572901.751202:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616311680:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572901.751217:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572901.751236:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572901/real 1788572901] req@ffff9ada4566c000 x1875449616311680/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572926 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572901.751247:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c000 x1875449616311680/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572926 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572901.751263:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572901.751267:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572901.751270:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616311680:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572901.751277:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616311808:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572901.751298:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572901.751311:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572901/real 1788572901] req@ffff9ada4566df80 x1875449616311808/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572926 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572901.751320:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566df80 x1875449616311808/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572926 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572901.751350:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572901.751352:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572901.751355:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616311808:192.168.202.113@tcp:38:kworker.0 00010000:00010000:1.0:1788572902.263092:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572903.287062:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572904.311027:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572905.335008:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572906.359018:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788572906.871003:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572906.871013:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788572906.871018:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572906.871030:0:91222:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4819 00000100:00080000:0.0:1788572906.871041:0:91222:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572906.871048:0:91222:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788572906.871083:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616311936/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788572906.871108:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572906.871111:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788572906.871113:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572906.871115:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4819 00000100:00080000:0.0:1788572906.871119:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572906.871122:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788572906.871132:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788572906.871146:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616312064/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788572906.871157:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1788572906.871161:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616311936:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788572906.871162:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788572906.871168:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572906.871174:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4819 00000100:00080000:0.0:1788572906.871179:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572906.871183:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788572906.871189:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788572906.871204:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616312192/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788572906.871222:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572906.871252:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572906/real 1788572906] req@ffff9ada4557dc00 x1875449616311936/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572931 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572906.871267:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616311936/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572931 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572906.871275:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572906.871278:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572906.871281:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616311936:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572906.871302:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616312064:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572906.871325:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572906.871338:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572906/real 1788572906] req@ffff9ada4557e300 x1875449616312064/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572931 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572906.871352:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616312064/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572931 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572906.871359:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572906.871361:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572906.871363:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616312064:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572906.871368:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616312192:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572906.871405:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572906.871414:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572906/real 1788572906] req@ffff9ada4557e680 x1875449616312192/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572931 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572906.871423:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616312192/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572931 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572906.871431:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572906.871434:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572906.871438:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616312192:192.168.202.113@tcp:38:kworker.0 00010000:00010000:0.0:1788572907.383028:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572908.407002:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572909.431022:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572910.455011:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572911.479044:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788572911.990943:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572911.990951:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788572911.990954:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572911.990967:0:91222:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4824 00000100:00080000:0.0:1788572911.990979:0:91222:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572911.990988:0:91222:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788572911.991028:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d180 x1875449616312320/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788572911.991060:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572911.991064:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788572911.991071:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572911.991075:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4824 00000100:00080000:0.0:1788572911.991080:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572911.991084:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788572911.991093:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788572911.991107:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616312448/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788572911.991117:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616312320:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788572911.991124:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572911.991128:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788572911.991130:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572911.991133:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4824 00000100:00080000:0.0:1788572911.991136:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572911.991138:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788572911.991142:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788572911.991152:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f100 x1875449616312576/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788572911.991191:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572911.991224:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572911/real 1788572911] req@ffff9ada4557d180 x1875449616312320/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572936 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572911.991240:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d180 x1875449616312320/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572936 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572911.991248:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572911.991251:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572911.991255:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616312320:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572911.991275:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616312448:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572911.991293:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572911.991301:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572911/real 1788572911] req@ffff9ada4557ed80 x1875449616312448/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572936 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572911.991310:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616312448/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572936 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572911.991316:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572911.991318:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572911.991320:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616312448:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572911.991324:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616312576:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572911.991334:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572911.991340:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572911/real 1788572911] req@ffff9ada4557f100 x1875449616312576/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572936 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572911.991348:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f100 x1875449616312576/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572936 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572911.991357:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572911.991359:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572911.991362:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616312576:192.168.202.113@tcp:38:kworker.0 00010000:00010000:1.0:1788572912.502987:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572913.527041:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572914.551019:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572915.575038:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572916.599035:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788572917.111000:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572917.111011:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788572917.111015:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572917.111025:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4829 00000100:00080000:1.0:1788572917.111035:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572917.111043:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788572917.111082:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566df80 x1875449616312704/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572917.111113:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572917.111118:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788572917.111120:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572917.111124:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4829 00000100:00080000:1.0:1788572917.111128:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572917.111132:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572917.111142:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572917.111156:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c000 x1875449616312832/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572917.111167:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572917.111171:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788572917.111174:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572917.111178:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4829 00000100:00080000:1.0:1788572917.111183:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572917.111187:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572917.111193:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572917.111203:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616312704:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788572917.111206:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566fb80 x1875449616312960/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788572917.111282:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616312832:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572917.111309:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616312960:192.168.202.113@tcp:38:kworker.0 00010000:00010000:1.0:1788572917.623078:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572918.647026:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572919.671101:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1788572920.183189:0:2372:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:0.0:1788572920.183235:0:2372:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1788572920.183253:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:0.0:1788572920.183295:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:0.0:1788572920.183306:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:3.0:1788572920.183349:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572917/real 1788572920] req@ffff9ada4566fb80 x1875449616312960/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572942 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572920.183376:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566fb80 x1875449616312960/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572942 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572920.183388:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572920.183393:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572920.183397:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616312960:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572920.183405:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572917/real 1788572920] req@ffff9ada4566df80 x1875449616312704/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572942 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572920.183418:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566df80 x1875449616312704/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572942 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572920.183432:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572920.183435:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572920.183438:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616312704:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572920.183444:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572917/real 1788572920] req@ffff9ada4566c000 x1875449616312832/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572942 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572920.183457:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c000 x1875449616312832/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572942 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572920.183466:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572920.183468:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572920.183472:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616312832:192.168.202.113@tcp:38:kworker.0 00010000:00010000:0.0:1788572920.695109:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572921.719086:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788572922.230988:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572922.230999:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788572922.231004:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572922.231015:0:91222:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4834 00000100:00080000:2.0:1788572922.231027:0:91222:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572922.231036:0:91222:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788572922.231075:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624700 x1875449616313088/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788572922.231116:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572922.231120:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788572922.231123:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572922.231127:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4834 00000100:00080000:2.0:1788572922.231131:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572922.231135:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572922.231146:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788572922.231160:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627100 x1875449616313216/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788572922.231171:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616313088:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788572922.231175:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572922.231181:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788572922.231183:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572922.231188:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4834 00000100:00080000:2.0:1788572922.231191:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572922.231201:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572922.231206:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788572922.231222:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616313344/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788572922.231252:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572922.231288:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572922/real 1788572922] req@ffff9ada60624700 x1875449616313088/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572947 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572922.231309:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624700 x1875449616313088/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572947 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572922.231321:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572922.231326:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572922.231330:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616313088:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572922.231355:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616313216:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572922.231391:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572922.231406:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572922/real 1788572922] req@ffff9ada60627100 x1875449616313216/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572947 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572922.231421:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627100 x1875449616313216/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572947 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572922.231430:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572922.231433:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572922.231437:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616313216:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572922.231445:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616313344:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572922.231468:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572922.231480:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572922/real 1788572922] req@ffff9ada60626680 x1875449616313344/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572947 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572922.231491:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616313344/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572947 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572922.231499:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572922.231503:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572922.231506:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616313344:192.168.202.113@tcp:38:kworker.0 00010000:00010000:3.0:1788572922.743031:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572923.767089:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572924.791006:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572925.815046:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572926.839022:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788572927.350986:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572927.350994:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788572927.350998:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572927.351006:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4839 00000100:00080000:1.0:1788572927.351015:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572927.351021:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788572927.351055:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566dc00 x1875449616313472/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572927.351077:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572927.351080:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788572927.351082:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572927.351085:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4839 00000100:00080000:1.0:1788572927.351087:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572927.351094:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572927.351102:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572927.351113:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f800 x1875449616313600/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572927.351128:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1788572927.351128:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616313472:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788572927.351133:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788572927.351137:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572927.351151:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4839 00000100:00080000:1.0:1788572927.351153:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572927.351155:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572927.351158:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572927.351170:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ed80 x1875449616313728/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788572927.351225:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572927.351268:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572927/real 1788572927] req@ffff9ada4566dc00 x1875449616313472/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572952 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572927.351287:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566dc00 x1875449616313472/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572952 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572927.351306:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572927.351309:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572927.351312:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616313472:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572927.351331:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616313600:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572927.351357:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572927.351367:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572927/real 1788572927] req@ffff9ada4566f800 x1875449616313600/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572952 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572927.351376:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f800 x1875449616313600/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572952 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572927.351382:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572927.351392:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572927.351395:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616313600:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572927.351398:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616313728:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572927.351410:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572927.351416:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572927/real 1788572927] req@ffff9ada4566ed80 x1875449616313728/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572952 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572927.351423:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ed80 x1875449616313728/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572952 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572927.351433:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572927.351435:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572927.351437:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616313728:192.168.202.113@tcp:38:kworker.0 00010000:00010000:1.0:1788572927.863020:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572928.887045:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572929.910980:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572930.935028:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572931.959060:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788572932.470977:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572932.470984:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788572932.470988:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572932.470996:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4845 00000100:00080000:1.0:1788572932.471007:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572932.471013:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788572932.471048:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ea00 x1875449616313856/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572932.471071:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572932.471074:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788572932.471075:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572932.471078:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4845 00000100:00080000:1.0:1788572932.471081:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572932.471083:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572932.471091:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572932.471100:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e300 x1875449616313984/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572932.471108:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572932.471111:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788572932.471113:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572932.471116:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4845 00000100:00080000:1.0:1788572932.471118:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572932.471121:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572932.471123:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572932.471132:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f480 x1875449616314112/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788572932.471150:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616313856:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788572932.471266:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572932.471329:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572932/real 1788572932] req@ffff9ada4566ea00 x1875449616313856/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572957 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572932.471356:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ea00 x1875449616313856/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572957 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572932.471369:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572932.471374:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572932.471379:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616313856:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572932.471389:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616313984:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572932.471416:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572932.471429:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572932/real 1788572932] req@ffff9ada4566e300 x1875449616313984/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572957 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572932.471440:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e300 x1875449616313984/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572957 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572932.471446:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572932.471450:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572932.471452:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616313984:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572932.471479:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616314112:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572932.471493:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572932.471501:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572932/real 1788572932] req@ffff9ada4566f480 x1875449616314112/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572957 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572932.471509:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f480 x1875449616314112/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572957 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572932.471515:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572932.471517:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572932.471519:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616314112:192.168.202.113@tcp:38:kworker.0 00010000:00010000:1.0:1788572932.983264:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572934.007001:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572935.031090:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572936.054985:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572937.079048:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788572937.590974:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572937.590986:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788572937.590990:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572937.591001:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4850 00000100:00080000:1.0:1788572937.591013:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572937.591021:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788572937.591066:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d180 x1875449616314240/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572937.591095:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572937.591098:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788572937.591100:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572937.591102:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4850 00000100:00080000:1.0:1788572937.591105:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572937.591107:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572937.591117:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572937.591126:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d500 x1875449616314368/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572937.591134:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572937.591137:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788572937.591139:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572937.591141:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4850 00000100:00080000:1.0:1788572937.591143:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572937.591145:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572937.591149:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572937.591152:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616314240:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788572937.591158:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ca80 x1875449616314496/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788572937.591239:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572937.591276:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572937/real 1788572937] req@ffff9ada4566d180 x1875449616314240/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572962 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572937.591293:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d180 x1875449616314240/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572962 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572937.591303:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572937.591306:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572937.591309:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616314240:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572937.591314:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616314368:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572937.591331:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572937.591339:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572937/real 1788572937] req@ffff9ada4566d500 x1875449616314368/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572962 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572937.591347:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d500 x1875449616314368/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572962 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572937.591352:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572937.591354:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572937.591356:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616314368:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572937.591378:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616314496:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572937.591389:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572937.591396:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572937/real 1788572937] req@ffff9ada4566ca80 x1875449616314496/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572962 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572937.591403:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ca80 x1875449616314496/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572962 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572937.591408:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572937.591410:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572937.591412:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616314496:192.168.202.113@tcp:38:kworker.0 00010000:00010000:3.0:1788572938.103071:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572939.127050:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572940.151025:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572941.174999:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572942.199020:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788572942.710946:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572942.710954:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788572942.710958:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572942.710966:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4855 00000100:00080000:2.0:1788572942.710975:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572942.710981:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788572942.711018:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616314624/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788572942.711047:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572942.711050:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788572942.711052:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788572942.711054:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4855 00000100:00080000:2.0:1788572942.711057:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572942.711060:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572942.711068:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788572942.711079:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616314752/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788572942.711087:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788572942.711089:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788572942.711093:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1788572942.711094:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616314624:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788572942.711098:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4855 00000100:00080000:2.0:1788572942.711103:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788572942.711106:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788572942.711110:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788572942.711118:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616314880/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788572942.711153:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572942.711182:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572942/real 1788572942] req@ffff9ada60625500 x1875449616314624/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572967 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572942.711196:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616314624/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572967 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572942.711204:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572942.711207:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572942.711210:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616314624:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572942.711229:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616314752:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572942.711242:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572942.711250:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572942/real 1788572942] req@ffff9ada60624380 x1875449616314752/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572967 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572942.711258:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616314752/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572967 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572942.711263:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572942.711265:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572942.711267:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616314752:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572942.711270:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616314880:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572942.711280:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572942.711285:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572942/real 1788572942] req@ffff9ada60626a00 x1875449616314880/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572967 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572942.711292:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616314880/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572967 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572942.711301:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572942.711303:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572942.711305:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616314880:192.168.202.113@tcp:38:kworker.0 00010000:00010000:3.0:1788572943.223041:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572944.247013:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572945.270983:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572946.295000:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572947.319132:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788572947.830950:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572947.830958:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788572947.830961:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572947.830970:0:91222:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4860 00000100:00080000:1.0:1788572947.830979:0:91222:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572947.830985:0:91222:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788572947.831019:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f100 x1875449616315008/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572947.831042:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572947.831044:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788572947.831046:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572947.831048:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4860 00000100:00080000:1.0:1788572947.831050:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572947.831053:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572947.831060:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572947.831069:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c380 x1875449616315136/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572947.831077:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572947.831079:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788572947.831081:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1788572947.831085:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616315008:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788572947.831094:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4860 00000100:00080000:1.0:1788572947.831097:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572947.831099:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572947.831105:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572947.831119:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c700 x1875449616315264/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788572947.831144:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572947.831174:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572947/real 1788572947] req@ffff9ada4566f100 x1875449616315008/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572972 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572947.831189:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f100 x1875449616315008/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572972 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572947.831197:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572947.831200:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572947.831202:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616315008:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572947.831220:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616315136:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572947.831235:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572947.831242:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572947/real 1788572947] req@ffff9ada4566c380 x1875449616315136/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572972 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572947.831250:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c380 x1875449616315136/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572972 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572947.831260:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572947.831262:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572947.831264:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616315136:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572947.831268:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616315264:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572947.831277:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572947.831282:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572947/real 1788572947] req@ffff9ada4566c700 x1875449616315264/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572972 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572947.831289:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c700 x1875449616315264/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572972 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572947.831294:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572947.831296:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572947.831298:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616315264:192.168.202.113@tcp:38:kworker.0 00010000:00010000:1.0:1788572948.343014:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572949.366991:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572950.390975:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572951.415045:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572952.439024:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788572952.950989:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572952.950998:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788572952.951001:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572952.951011:0:91222:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4865 00000100:00080000:0.0:1788572952.951021:0:91222:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572952.951029:0:91222:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788572952.951071:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557df80 x1875449616315392/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788572952.951101:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572952.951105:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788572952.951107:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572952.951112:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4865 00000100:00080000:0.0:1788572952.951116:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572952.951120:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788572952.951131:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788572952.951144:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616315520/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788572952.951150:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616315392:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788572952.951156:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572952.951160:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788572952.951162:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572952.951165:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4865 00000100:00080000:0.0:1788572952.951168:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572952.951171:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788572952.951175:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788572952.951185:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d500 x1875449616315648/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788572952.951224:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572952.951261:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572952/real 1788572952] req@ffff9ada4557df80 x1875449616315392/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572977 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572952.951280:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557df80 x1875449616315392/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572977 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572952.951299:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572952.951303:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572952.951307:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616315392:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572952.951338:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616315520:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572952.951358:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572952.951368:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572952/real 1788572952] req@ffff9ada4557ea00 x1875449616315520/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572977 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572952.951377:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616315520/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572977 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572952.951382:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572952.951384:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572952.951387:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616315520:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572952.951391:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616315648:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572952.951401:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572952.951407:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572952/real 1788572952] req@ffff9ada4557d500 x1875449616315648/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572977 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572952.951415:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d500 x1875449616315648/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572977 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572952.951420:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572952.951422:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572952.951424:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616315648:192.168.202.113@tcp:38:kworker.0 00010000:00010000:3.0:1788572953.463005:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572954.486983:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572955.511050:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572956.535019:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572957.559028:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788572958.070954:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572958.070962:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788572958.070966:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572958.070975:0:91222:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4870 00000100:00080000:0.0:1788572958.070984:0:91222:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572958.070990:0:91222:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788572958.071025:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d500 x1875449616315776/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788572958.071048:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572958.071051:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788572958.071052:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572958.071055:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4870 00000100:00080000:0.0:1788572958.071057:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572958.071059:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788572958.071067:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788572958.071074:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616315904/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788572958.071084:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572958.071087:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788572958.071090:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572958.071094:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4870 00000100:00080000:0.0:1788572958.071099:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572958.071103:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788572958.071109:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572958.071113:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616315776:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788572958.071123:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557df80 x1875449616316032/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788572958.071179:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572958.071227:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572958/real 1788572958] req@ffff9ada4557d500 x1875449616315776/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572983 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572958.071256:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d500 x1875449616315776/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572983 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572958.071268:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572958.071273:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572958.071277:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616315776:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572958.071284:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616315904:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572958.071310:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572958.071325:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572958/real 1788572958] req@ffff9ada4557ea00 x1875449616315904/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572983 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572958.071339:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616315904/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572983 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572958.071349:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572958.071352:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572958.071356:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616315904:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572958.071393:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616316032:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572958.071417:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572958.071430:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572958/real 1788572958] req@ffff9ada4557df80 x1875449616316032/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572983 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572958.071444:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557df80 x1875449616316032/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572983 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572958.071452:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572958.071456:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572958.071459:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616316032:192.168.202.113@tcp:38:kworker.0 00010000:00010000:3.0:1788572958.583197:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572959.607042:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572960.631040:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572961.655345:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572962.679085:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788572963.190994:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572963.191002:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788572963.191006:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572963.191014:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4875 00000100:00080000:3.0:1788572963.191023:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572963.191030:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788572963.191062:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1880 x1875449616316160/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572963.191087:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572963.191090:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788572963.191091:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572963.191094:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4875 00000100:00080000:3.0:1788572963.191097:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572963.191099:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572963.191107:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572963.191116:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1180 x1875449616316288/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572963.191125:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572963.191127:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788572963.191128:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572963.191133:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4875 00000100:00100000:1.0:1788572963.191136:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616316160:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788572963.191136:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572963.191142:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572963.191148:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572963.191155:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3800 x1875449616316416/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788572963.191209:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572963.191250:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572963/real 1788572963] req@ffff9ada604e1880 x1875449616316160/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572988 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572963.191271:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1880 x1875449616316160/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572988 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572963.191282:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572963.191286:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572963.191290:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616316160:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788572963.191315:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616316288:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572963.191340:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572963.191350:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572963/real 1788572963] req@ffff9ada604e1180 x1875449616316288/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572988 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572963.191358:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1180 x1875449616316288/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572988 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572963.191364:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572963.191366:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572963.191369:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616316288:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572963.191374:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616316416:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572963.191385:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572963.191391:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572963/real 1788572963] req@ffff9ada604e3800 x1875449616316416/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572988 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572963.191399:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3800 x1875449616316416/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572988 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572963.191404:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572963.191405:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572963.191407:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616316416:192.168.202.113@tcp:38:kworker.0 00010000:00010000:1.0:1788572963.703015:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572964.727018:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572965.750982:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572966.775030:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572967.799055:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788572968.310985:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572968.310993:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788572968.310998:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572968.311007:0:91222:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4880 00000100:00080000:1.0:1788572968.311018:0:91222:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572968.311026:0:91222:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788572968.311060:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ce00 x1875449616316544/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572968.311081:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572968.311085:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788572968.311087:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572968.311090:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4880 00000100:00080000:1.0:1788572968.311093:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572968.311097:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572968.311107:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572968.311116:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616316544:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788572968.311125:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d880 x1875449616316672/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788572968.311139:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788572968.311143:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788572968.311145:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788572968.311150:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4880 00000100:00080000:1.0:1788572968.311154:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788572968.311158:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788572968.311164:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572968.311178:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e680 x1875449616316800/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788572968.311187:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572968.311232:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572968/real 1788572968] req@ffff9ada4566ce00 x1875449616316544/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572993 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572968.311250:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ce00 x1875449616316544/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572993 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572968.311259:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572968.311262:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572968.311267:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616316544:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572968.311286:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616316672:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572968.311306:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572968.311315:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572968/real 1788572968] req@ffff9ada4566d880 x1875449616316672/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572993 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572968.311324:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d880 x1875449616316672/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572993 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572968.311329:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572968.311332:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572968.311343:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616316672:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788572968.311348:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616316800:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788572968.311363:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572968.311371:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572968/real 1788572968] req@ffff9ada4566e680 x1875449616316800/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572993 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788572968.311379:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e680 x1875449616316800/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572993 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788572968.311384:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788572968.311386:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788572968.311389:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616316800:192.168.202.113@tcp:38:kworker.0 00000001:02000400:2.0:1788572968.733262:0:93070:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == replay-single test 44c: race in target handle connect ========================================================== 21:49:28 (1788572968) 00010000:00010000:3.0:1788572968.823063:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572969.846999:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788572970.870993:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788572971.895037:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788572972.918985:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788572973.430968:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572973.430980:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788572973.430984:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572973.430994:0:91222:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4885 00000100:00080000:3.0:1788572973.431004:0:91222:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572973.431012:0:91222:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788572973.431050:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2300 x1875449616316928/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572973.431074:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572973.431077:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788572973.431079:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572973.431082:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4885 00000100:00080000:3.0:1788572973.431085:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572973.431089:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572973.431099:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572973.431106:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616316928:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788572973.431116:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2d80 x1875449616317056/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572973.431130:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572973.431134:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788572973.431137:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572973.431141:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4885 00000100:00080000:3.0:1788572973.431146:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572973.431150:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572973.431155:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000800:00000100:1.0:1788572973.431165:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788572973.431168:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0a80 x1875449616317184/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788572973.431202:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572973/real 1788572973] req@ffff9ada604e2300 x1875449616316928/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572998 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572973.431217:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2300 x1875449616316928/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788572998 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572973.431229:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572973.431232:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572973.431235:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616316928:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788572973.431253:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616317056:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572973.431268:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572973.431276:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572973/real 1788572973] req@ffff9ada604e2d80 x1875449616317056/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572998 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572973.431284:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2d80 x1875449616317056/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572998 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572973.431289:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572973.431291:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572973.431294:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616317056:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572973.431297:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616317184:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572973.431307:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572973.431313:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572973/real 1788572973] req@ffff9ada604e0a80 x1875449616317184/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572998 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572973.431320:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0a80 x1875449616317184/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788572998 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572973.431325:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572973.431327:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572973.431329:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616317184:192.168.202.113@tcp:38:kworker.0 00010000:00010000:1.0:1788572973.943040:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:0.0:1788572974.656337:0:93093:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1788572974.656351:0:93093:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572974.656374:0:93093:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572974.656395:0:93093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 76 lru_type: 1 00010000:00010000:0.0:1788572974.656412:0:93093:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 76 lru_type: 1 00000080:00200000:0.0:1788572974.656422:0:93093:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572974.656429:0:93093:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572974.656433:0:93093:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:0.0:1788572974.656441:0:93093:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec16d3cb8 00010000:00010000:0.0:1788572974.656445:0:93093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 76 lru_type: 1 00010000:00010000:0.0:1788572974.656455:0:93093:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 77 lru_type: 1 00000080:00200008:0.0:1788572974.657031:0:93093:0:(file.c:6296:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1788572974.657037:0:93093:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572974.657047:0:93093:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572974.657056:0:93093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 77 lru_type: 1 00010000:00010000:0.0:1788572974.657067:0:93093:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 77 lru_type: 1 00000080:00200000:0.0:1788572974.657082:0:93093:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572974.657087:0:93093:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572974.657090:0:93093:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:0.0:1788572974.657095:0:93093:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec16d3cb8 00010000:00010000:0.0:1788572974.657099:0:93093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 77 lru_type: 1 00010000:00010000:0.0:1788572974.657108:0:93093:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 78 lru_type: 1 00000080:00200000:0.0:1788572974.657182:0:93093:0:(file.c:6191:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88),name=/ 00000002:00010000:0.0:1788572974.657191:0:93093:0:(mdc_locks.c:1508:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1788572974.657198:0:93093:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 78 lru_type: 1 00010000:00010000:0.0:1788572974.657208:0:93093:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 78 lru_type: 1 00000080:00200000:0.0:1788572974.657215:0:93093:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1788572974.657219:0:93093:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) 00000080:00010000:0.0:1788572974.657222:0:93093:0:(llite_internal.h:1989:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff9ada58125b88) for lock 0xef30c340bfb4b27d 00000080:00010000:0.0:1788572974.657226:0:93093:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0xef30c340bfb4b27d from it ffffab0ec16d3bb0 00010000:00010000:0.0:1788572974.657230:0:93093:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 78 lru_type: 1 00010000:00010000:0.0:1788572974.657237:0:93093:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff9ada58b0a000 lock: ffff9ada455a8a00/0xef30c340bfb4b27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xdb7edb52b833a412 expref: -99 pid: 90306 timeout: 0 lvb_type: 0 lru_score: 79 lru_type: 1 00000080:00200000:0.0:1788572974.657247:0:93093:0:(file.c:6672:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88), inode mode 41ed mask 44 00000080:00200008:0.0:1788572974.657254:0:93093:0:(file.c:1090:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff9ada58125b88), flags 100000 00000080:00200080:0.0:1788572974.657329:0:93093:0:(dir.c:2004:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff9ada58125b88) cmd=c00866a4 arg=7ffdd33accf0 00800000:00000080:0.0:1788572974.657345:0:93093:0:(lmv_obd.c:934:lmv_iocontrol()) lustre-clilmv-ffff9ada58b0a000: cmd=c00866a4 len=592 karg=ffff9ada45571800 uarg=0000000000000000 00800000:00000004:0.0:1788572974.657353:0:93093:0:(obd_class.h:1090:obd_statfs()) lustre-MDT0000-mdc-ffff9ada58b0a000: age 4488, max_age 4891 00000100:00080000:0.0:1788572974.657399:0:93093:0:(client.c:1846:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != DISCONN) req@ffff9ada4557f100 x1875449616317312/t0(0) o41->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788572974.657411:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788572974.967152:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572975.670952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788572975.991033:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572976.694963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788572977.015048:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572977.718973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788572978.039010:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788572978.550984:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572978.550992:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788572978.550996:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572978.551005:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4891 00000100:00080000:0.0:1788572978.551014:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572978.551020:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788572978.551053:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616317440/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788572978.551075:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572978.551077:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788572978.551079:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572978.551081:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4891 00000100:00080000:0.0:1788572978.551084:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572978.551087:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788572978.551094:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788572978.551102:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d180 x1875449616317568/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788572978.551109:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572978.551113:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788572978.551114:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572978.551117:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4891 00000100:00080000:0.0:1788572978.551119:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572978.551121:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788572978.551124:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788572978.551132:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616317696/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788572978.551134:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616317440:192.168.202.113@tcp:250:kworker.0 00000800:00000100:1.0:1788572978.551202:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572978.551240:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572978/real 1788572978] req@ffff9ada4557ed80 x1875449616317440/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573003 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572978.551260:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616317440/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573003 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572978.551272:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572978.551277:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572978.551281:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616317440:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788572978.551289:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616317568:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572978.551318:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572978.551330:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572978/real 1788572978] req@ffff9ada4557d180 x1875449616317568/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573003 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572978.551343:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d180 x1875449616317568/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573003 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572978.551353:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572978.551356:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572978.551360:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616317568:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572978.551392:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616317696:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572978.551415:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572978.551426:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572978/real 1788572978] req@ffff9ada4557e680 x1875449616317696/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573003 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572978.551440:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616317696/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573003 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572978.551448:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572978.551452:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572978.551456:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616317696:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788572978.742977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788572979.063034:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572979.766943:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788572980.087044:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572980.790964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788572981.111033:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572981.814964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788572982.135103:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572982.838995:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788572983.159053:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788572983.671017:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572983.671029:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788572983.671033:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572983.671044:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4896 00000100:00080000:3.0:1788572983.671060:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572983.671069:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788572983.671113:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3b80 x1875449616317824/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572983.671136:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572983.671139:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788572983.671141:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572983.671144:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4896 00000100:00080000:3.0:1788572983.671146:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572983.671149:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572983.671159:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572983.671168:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0000 x1875449616317952/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572983.671176:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572983.671181:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:1.0:1788572983.671183:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616317824:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788572983.671184:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572983.671192:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4896 00000100:00080000:3.0:1788572983.671195:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572983.671200:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572983.671205:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572983.671220:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1c00 x1875449616318080/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788572983.671296:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616317952:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572983.671332:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616318080:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788572983.862967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788572984.183050:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572984.886994:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788572985.207001:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572985.910940:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788572986.231196:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1788572986.743168:0:2373:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:2.0:1788572986.743215:0:2373:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1788572986.743237:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788572986.743281:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788572986.743290:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:1.0:1788572986.743343:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572983/real 1788572986] req@ffff9ada604e0000 x1875449616317952/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573008 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572986.743372:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0000 x1875449616317952/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573008 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572986.743386:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572986.743393:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572986.743398:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616317952:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572986.743414:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572983/real 1788572986] req@ffff9ada604e1c00 x1875449616318080/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573008 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572986.743429:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1c00 x1875449616318080/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573008 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572986.743438:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572986.743443:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572986.743446:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616318080:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572986.743468:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572983/real 1788572986] req@ffff9ada604e3b80 x1875449616317824/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573008 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572986.743481:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3b80 x1875449616317824/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573008 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572986.743490:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572986.743494:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572986.743497:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616317824:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788572986.934975:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788572987.255115:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572987.958940:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788572988.279003:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788572988.791011:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572988.791023:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788572988.791027:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572988.791038:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4901 00000100:00080000:0.0:1788572988.791050:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572988.791058:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788572988.791098:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616318208/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788572988.791131:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572988.791135:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788572988.791137:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572988.791142:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4901 00000100:00080000:0.0:1788572988.791146:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572988.791150:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788572988.791163:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788572988.791180:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616318336/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788572988.791192:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788572988.791196:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788572988.791198:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788572988.791203:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4901 00000100:00080000:0.0:1788572988.791208:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788572988.791214:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788572988.791226:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788572988.791226:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616318208:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788572988.791251:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616318464/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788572988.791315:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572988.791360:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572988/real 1788572988] req@ffff9ada4557e300 x1875449616318208/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573013 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572988.791382:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616318208/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573013 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572988.791404:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572988.791409:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572988.791414:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616318208:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788572988.791423:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616318336:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572988.791450:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572988.791465:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572988/real 1788572988] req@ffff9ada4557dc00 x1875449616318336/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573013 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572988.791480:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616318336/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573013 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572988.791489:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572988.791493:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572988.791496:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616318336:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572988.791525:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616318464:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572988.791548:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572988.791562:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572988/real 1788572988] req@ffff9ada4557fb80 x1875449616318464/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573013 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572988.791574:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616318464/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573013 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572988.791581:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572988.791585:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572988.791625:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616318464:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788572988.982979:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788572989.303041:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572990.007005:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788572990.327043:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572991.031001:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788572991.351037:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572992.054978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788572992.375005:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572993.078944:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788572993.399053:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788572993.910974:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572993.910983:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788572993.910986:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572993.910995:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4906 00000100:00080000:3.0:1788572993.911004:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572993.911009:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788572993.911045:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3100 x1875449616318592/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572993.911080:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572993.911084:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788572993.911086:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572993.911090:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4906 00000100:00080000:3.0:1788572993.911094:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572993.911098:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572993.911109:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572993.911127:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1f80 x1875449616318720/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788572993.911128:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616318592:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788572993.911147:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572993.911150:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788572993.911152:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572993.911156:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4906 00000100:00080000:3.0:1788572993.911160:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572993.911164:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572993.911169:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572993.911182:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2680 x1875449616318848/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788572993.911192:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572993.911226:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572993/real 1788572993] req@ffff9ada604e3100 x1875449616318592/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573018 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572993.911241:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3100 x1875449616318592/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573018 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572993.911249:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572993.911251:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572993.911254:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616318592:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788572993.911272:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616318720:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572993.911286:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572993.911294:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572993/real 1788572993] req@ffff9ada604e1f80 x1875449616318720/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573018 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572993.911302:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1f80 x1875449616318720/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573018 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572993.911308:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572993.911310:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572993.911312:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616318720:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572993.911316:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616318848:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572993.911326:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572993.911332:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572993/real 1788572993] req@ffff9ada604e2680 x1875449616318848/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573018 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572993.911339:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2680 x1875449616318848/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573018 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572993.911344:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572993.911346:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572993.911347:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616318848:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788572994.103017:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788572994.423057:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572995.126962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788572995.447043:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572996.150964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788572996.471047:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572997.174950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788572997.494959:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788572998.198986:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788572998.519036:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788572999.030956:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572999.030964:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788572999.030968:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572999.030976:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4911 00000100:00080000:3.0:1788572999.030985:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572999.030990:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788572999.031022:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1500 x1875449616318976/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572999.031045:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788572999.031049:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788572999.031052:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572999.031056:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4911 00000100:00080000:3.0:1788572999.031060:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572999.031063:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572999.031074:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572999.031087:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0e00 x1875449616319104/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788572999.031097:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1788572999.031098:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616318976:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788572999.031099:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788572999.031101:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788572999.031105:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4911 00000100:00080000:3.0:1788572999.031108:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788572999.031111:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788572999.031115:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788572999.031124:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2a00 x1875449616319232/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788572999.031155:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572999.031196:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572999/real 1788572999] req@ffff9ada604e1500 x1875449616318976/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573024 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572999.031210:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1500 x1875449616318976/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573024 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572999.031218:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572999.031220:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572999.031223:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616318976:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788572999.031243:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616319104:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572999.031271:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572999.031285:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572999/real 1788572999] req@ffff9ada604e0e00 x1875449616319104/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573024 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572999.031303:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0e00 x1875449616319104/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573024 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572999.031309:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572999.031311:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572999.031313:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616319104:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788572999.031317:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616319232:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788572999.031332:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788572999.031339:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788572999/real 1788572999] req@ffff9ada604e2a00 x1875449616319232/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573024 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788572999.031346:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2a00 x1875449616319232/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573024 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788572999.031351:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788572999.031353:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788572999.031355:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616319232:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788572999.222948:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788572999.543072:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788573000.246986:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573000.566995:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788573001.270951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573001.590943:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788573002.294971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573002.615037:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788573003.318938:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573003.639026:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573004.150984:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573004.150993:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573004.150997:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573004.151005:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4916 00000100:00080000:3.0:1788573004.151015:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573004.151021:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573004.151063:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2a00 x1875449616319360/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573004.151097:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573004.151101:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573004.151103:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573004.151108:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4916 00000100:00080000:3.0:1788573004.151112:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573004.151117:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573004.151127:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573004.151142:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0e00 x1875449616319488/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573004.151152:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573004.151157:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:1.0:1788573004.151158:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616319360:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788573004.151159:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573004.151165:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4916 00000100:00080000:3.0:1788573004.151171:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573004.151174:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573004.151178:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573004.151189:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1500 x1875449616319616/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573004.151241:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573004.151278:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573004/real 1788573004] req@ffff9ada604e2a00 x1875449616319360/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573029 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573004.151298:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2a00 x1875449616319360/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573029 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573004.151308:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573004.151313:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573004.151317:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616319360:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573004.151339:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616319488:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573004.151361:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573004.151374:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573004/real 1788573004] req@ffff9ada604e0e00 x1875449616319488/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573029 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573004.151396:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0e00 x1875449616319488/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573029 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573004.151404:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573004.151407:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573004.151410:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616319488:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573004.151417:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616319616:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573004.151435:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573004.151446:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573004/real 1788573004] req@ffff9ada604e1500 x1875449616319616/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573029 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573004.151457:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1500 x1875449616319616/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573029 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573004.151470:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573004.151474:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573004.151477:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616319616:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788573004.342984:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573004.663005:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788573005.366965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573005.687036:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788573006.390950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573006.711132:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788573007.414946:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573007.735088:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788573008.438984:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573008.759057:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573009.270993:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573009.271004:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573009.271009:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573009.271021:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4921 00000100:00080000:0.0:1788573009.271033:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573009.271041:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573009.271095:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616319744/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573009.271127:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573009.271131:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573009.271133:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573009.271138:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4921 00000100:00080000:0.0:1788573009.271142:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573009.271148:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573009.271161:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573009.271171:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616319744:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788573009.271179:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616319872/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573009.271191:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573009.271195:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573009.271197:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573009.271201:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4921 00000100:00080000:0.0:1788573009.271205:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573009.271210:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573009.271215:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573009.271228:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616320000/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573009.271247:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573009.271288:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573009/real 1788573009] req@ffff9ada4557d880 x1875449616319744/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573034 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573009.271305:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616319744/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573034 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573009.271317:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573009.271322:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573009.271326:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616319744:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573009.271354:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616319872:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573009.271377:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573009.271387:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573009/real 1788573009] req@ffff9ada4557f800 x1875449616319872/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573034 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573009.271396:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616319872/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573034 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573009.271402:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573009.271405:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573009.271408:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616319872:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573009.271412:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616320000:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573009.271424:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573009.271431:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573009/real 1788573009] req@ffff9ada4557ca80 x1875449616320000/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573034 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573009.271438:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616320000/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573034 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573009.271443:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573009.271445:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573009.271447:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616320000:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788573009.463035:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573009.783111:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788573010.486986:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573010.807046:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788573011.510942:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573011.830983:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788573012.534944:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573012.855026:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788573013.558981:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573013.878998:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573014.390959:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573014.390968:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573014.390971:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573014.390986:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4926 00000100:00080000:3.0:1788573014.390995:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573014.391001:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573014.391034:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2680 x1875449616320128/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573014.391058:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573014.391060:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573014.391062:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573014.391065:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4926 00000100:00080000:3.0:1788573014.391067:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573014.391069:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573014.391076:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573014.391086:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1f80 x1875449616320256/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573014.391094:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573014.391096:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573014.391098:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573014.391102:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4926 00000100:00100000:1.0:1788573014.391104:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616320128:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788573014.391105:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573014.391112:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573014.391116:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573014.391124:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3100 x1875449616320384/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573014.391166:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573014.391202:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573014/real 1788573014] req@ffff9ada604e2680 x1875449616320128/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573039 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573014.391223:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2680 x1875449616320128/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573039 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573014.391234:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573014.391239:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573014.391243:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616320128:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573014.391265:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616320256:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573014.391289:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573014.391304:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573014/real 1788573014] req@ffff9ada604e1f80 x1875449616320256/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573039 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573014.391316:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1f80 x1875449616320256/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573039 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573014.391325:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573014.391329:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573014.391332:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616320256:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573014.391339:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616320384:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573014.391360:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573014.391389:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573014/real 1788573014] req@ffff9ada604e3100 x1875449616320384/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573039 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573014.391398:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3100 x1875449616320384/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573039 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573014.391404:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573014.391408:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573014.391411:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616320384:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788573014.582937:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573014.902977:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788573015.606964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573015.927052:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788573016.630960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573016.950995:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788573017.654964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573017.975023:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788573018.678969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573018.999027:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573019.510924:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573019.510932:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573019.510936:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573019.510944:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4932 00000100:00080000:3.0:1788573019.510953:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573019.510959:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573019.510991:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3b80 x1875449616320512/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573019.511013:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573019.511016:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573019.511018:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573019.511021:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4932 00000100:00080000:3.0:1788573019.511025:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573019.511029:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573019.511038:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573019.511053:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1c00 x1875449616320640/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573019.511060:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616320512:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788573019.511071:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573019.511076:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573019.511078:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573019.511082:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4932 00000100:00080000:3.0:1788573019.511085:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573019.511088:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573019.511094:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573019.511104:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0000 x1875449616320768/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573019.511124:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573019.511155:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573019/real 1788573019] req@ffff9ada604e3b80 x1875449616320512/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573044 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573019.511170:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3b80 x1875449616320512/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573044 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573019.511178:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573019.511180:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573019.511183:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616320512:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573019.511203:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616320640:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573019.511217:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573019.511225:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573019/real 1788573019] req@ffff9ada604e1c00 x1875449616320640/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573044 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573019.511232:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1c00 x1875449616320640/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573044 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573019.511238:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573019.511240:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573019.511242:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616320640:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573019.511245:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616320768:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573019.511255:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573019.511260:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573019/real 1788573019] req@ffff9ada604e0000 x1875449616320768/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573044 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573019.511291:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0000 x1875449616320768/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573044 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573019.511296:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573019.511298:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573019.511301:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616320768:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788573019.702973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573020.023014:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788573020.726946:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573021.047032:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788573021.750962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573022.070981:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788573022.774964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573023.095022:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788573023.798943:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573024.118974:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573024.630964:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573024.630973:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573024.630976:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573024.630985:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4937 00000100:00080000:0.0:1788573024.630994:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573024.631000:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573024.631033:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c000 x1875449616320896/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573024.631055:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573024.631058:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573024.631060:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573024.631062:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4937 00000100:00080000:0.0:1788573024.631065:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573024.631068:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573024.631076:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573024.631085:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616321024/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573024.631094:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616320896:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788573024.631095:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573024.631104:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573024.631106:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573024.631109:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4937 00000100:00080000:0.0:1788573024.631111:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573024.631113:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573024.631117:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573024.631125:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616321152/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573024.631156:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573024.631210:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573024/real 1788573024] req@ffff9ada4557c000 x1875449616320896/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573049 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573024.631225:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c000 x1875449616320896/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573049 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573024.631233:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573024.631236:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573024.631239:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616320896:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573024.631258:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616321024:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573024.631279:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573024.631292:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573024/real 1788573024] req@ffff9ada4557f480 x1875449616321024/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573049 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573024.631315:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616321024/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573049 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573024.631322:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573024.631325:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573024.631327:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616321024:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573024.631331:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616321152:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573024.631357:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573024.631365:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573024/real 1788573024] req@ffff9ada4557c700 x1875449616321152/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573049 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573024.631373:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616321152/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573049 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573024.631379:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573024.631381:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573024.631383:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616321152:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788573024.822946:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573025.143013:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788573025.846955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573026.167054:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788573026.870928:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573027.191011:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788573027.894950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573028.215100:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788573028.918988:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573029.239011:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573029.750937:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573029.750948:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573029.750953:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573029.750963:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4942 00000100:00080000:0.0:1788573029.750974:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573029.750982:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573029.751014:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c380 x1875449616321280/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573029.751037:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573029.751041:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573029.751043:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573029.751047:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4942 00000100:00080000:0.0:1788573029.751052:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573029.751056:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573029.751066:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573029.751077:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ce00 x1875449616321408/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573029.751090:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616321280:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788573029.751100:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573029.751103:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573029.751104:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573029.751108:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4942 00000100:00080000:0.0:1788573029.751111:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573029.751114:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573029.751119:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573029.751130:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616321536/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573029.751150:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573029.751193:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573029/real 1788573029] req@ffff9ada4557c380 x1875449616321280/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573054 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573029.751212:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c380 x1875449616321280/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573054 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573029.751220:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573029.751223:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573029.751225:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616321280:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573029.751244:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616321408:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573029.751261:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573029.751270:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573029/real 1788573029] req@ffff9ada4557ce00 x1875449616321408/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573054 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573029.751278:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ce00 x1875449616321408/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573054 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573029.751284:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573029.751285:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573029.751287:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616321408:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573029.751291:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616321536:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573029.751301:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573029.751308:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573029/real 1788573029] req@ffff9ada4557c700 x1875449616321536/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573054 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573029.751315:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616321536/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573054 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573029.751324:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573029.751326:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573029.751328:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616321536:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788573029.942991:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573030.263016:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788573030.966951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573031.286978:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788573031.990946:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573032.311020:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788573033.014981:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573033.335054:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788573034.038984:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573034.359061:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573034.870963:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573034.870972:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573034.870976:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573034.870984:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4947 00000100:00080000:0.0:1788573034.871014:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573034.871020:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573034.871054:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616321664/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573034.871076:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573034.871078:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573034.871080:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573034.871082:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4947 00000100:00080000:0.0:1788573034.871085:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573034.871087:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573034.871094:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573034.871102:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c000 x1875449616321792/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573034.871109:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573034.871111:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573034.871113:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573034.871116:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4947 00000100:00080000:0.0:1788573034.871119:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573034.871122:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573034.871125:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573034.871134:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616321920/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573034.871143:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616321664:192.168.202.113@tcp:250:kworker.0 00000800:00000100:1.0:1788573034.871218:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573034.871254:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573034/real 1788573034] req@ffff9ada4557f480 x1875449616321664/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573059 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573034.871274:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616321664/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573059 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573034.871285:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573034.871289:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573034.871293:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616321664:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573034.871300:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616321792:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573034.871323:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573034.871336:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573034/real 1788573034] req@ffff9ada4557c000 x1875449616321792/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573059 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573034.871350:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c000 x1875449616321792/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573059 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573034.871359:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573034.871363:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573034.871367:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616321792:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573034.871391:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616321920:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573034.871411:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573034.871423:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573034/real 1788573034] req@ffff9ada4557ca80 x1875449616321920/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573059 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573034.871436:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616321920/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573059 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573034.871445:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573034.871449:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573034.871452:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616321920:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788573035.062998:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573035.383018:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573036.086978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573036.407070:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573037.110993:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573037.431080:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573038.134967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573038.455088:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573039.159019:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573039.479042:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573039.991012:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573039.991028:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573039.991035:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573039.991048:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4952 00000100:00080000:3.0:1788573039.991061:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573039.991072:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573039.991124:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0a80 x1875449616322048/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573039.991160:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573039.991165:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573039.991167:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573039.991172:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4952 00000100:00080000:3.0:1788573039.991177:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573039.991182:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573039.991195:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573039.991209:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616322048:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573039.991213:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2d80 x1875449616322176/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573039.991228:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573039.991232:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573039.991235:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573039.991239:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4952 00000100:00080000:3.0:1788573039.991245:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573039.991250:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573039.991266:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573039.991282:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2300 x1875449616322304/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573039.991286:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573039.991342:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573039/real 1788573039] req@ffff9ada604e0a80 x1875449616322048/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573064 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573039.991359:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0a80 x1875449616322048/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573064 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573039.991369:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573039.991372:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573039.991375:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616322048:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573039.991398:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616322176:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573039.991415:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573039.991424:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573039/real 1788573039] req@ffff9ada604e2d80 x1875449616322176/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573064 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573039.991433:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2d80 x1875449616322176/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573064 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573039.991439:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573039.991440:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573039.991442:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616322176:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573039.991446:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616322304:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573039.991458:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573039.991464:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573039/real 1788573039] req@ffff9ada604e2300 x1875449616322304/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573064 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573039.991472:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2300 x1875449616322304/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573064 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573039.991482:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573039.991484:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573039.991486:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616322304:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573040.183002:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573040.503074:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573041.206985:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573041.527015:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573042.230935:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573042.551072:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573043.254962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573043.574999:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573044.278940:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573044.599030:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573045.110996:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573045.111004:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573045.111008:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573045.111017:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4957 00000100:00080000:2.0:1788573045.111025:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573045.111032:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573045.111065:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616322432/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573045.111088:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573045.111091:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573045.111092:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573045.111095:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4957 00000100:00080000:2.0:1788573045.111098:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573045.111100:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573045.111108:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573045.111117:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616322560/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573045.111125:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573045.111128:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573045.111129:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1788573045.111134:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616322432:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788573045.111135:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4957 00000100:00080000:2.0:1788573045.111140:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573045.111145:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573045.111148:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573045.111158:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616322688/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573045.111204:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573045.111236:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573045/real 1788573045] req@ffff9ada60624a80 x1875449616322432/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573070 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573045.111250:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616322432/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573070 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573045.111258:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573045.111261:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573045.111264:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616322432:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573045.111288:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616322560:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573045.111303:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573045.111312:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573045/real 1788573045] req@ffff9ada60625880 x1875449616322560/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573070 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573045.111321:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616322560/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573070 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573045.111326:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573045.111328:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573045.111331:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616322560:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573045.111335:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616322688:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573045.111345:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573045.111351:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573045/real 1788573045] req@ffff9ada60625c00 x1875449616322688/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573070 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573045.111358:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616322688/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573070 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573045.111363:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573045.111365:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573045.111367:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616322688:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573045.302949:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573045.623089:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573046.326979:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573046.647031:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573047.350945:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573047.671016:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573048.375013:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573048.695023:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573049.398946:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573049.719085:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573050.230963:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573050.230974:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573050.230979:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573050.230991:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4962 00000100:00080000:3.0:1788573050.231003:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573050.231011:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573050.231055:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3800 x1875449616322816/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573050.231084:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573050.231088:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573050.231090:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573050.231092:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4962 00000100:00080000:3.0:1788573050.231095:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573050.231097:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573050.231106:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573050.231116:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1180 x1875449616322944/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573050.231126:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573050.231129:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573050.231131:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1788573050.231137:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616322816:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788573050.231137:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4962 00000100:00080000:3.0:1788573050.231143:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573050.231149:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573050.231153:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573050.231161:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1880 x1875449616323072/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573050.231250:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616322944:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573050.231274:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616323072:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573050.423038:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573050.743087:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573051.446944:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573051.767030:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573052.470964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573052.791024:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1788573053.303112:0:2371:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:0.0:1788573053.303148:0:2371:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1788573053.303161:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:0.0:1788573053.303195:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:0.0:1788573053.303202:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:1.0:1788573053.303251:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573050/real 1788573053] req@ffff9ada604e1180 x1875449616322944/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573075 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573053.303278:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1180 x1875449616322944/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573075 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573053.303291:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573053.303296:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573053.303301:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616322944:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573053.303310:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573050/real 1788573053] req@ffff9ada604e1880 x1875449616323072/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573075 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573053.303323:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1880 x1875449616323072/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573075 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573053.303331:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573053.303335:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573053.303338:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616323072:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573053.303344:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573050/real 1788573053] req@ffff9ada604e3800 x1875449616322816/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573075 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573053.303365:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3800 x1875449616322816/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573075 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573053.303373:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573053.303378:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573053.303382:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616322816:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573053.494936:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573053.815026:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573054.518940:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573054.839008:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573055.351019:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573055.351030:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573055.351035:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573055.351046:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4967 00000100:00080000:0.0:1788573055.351057:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573055.351067:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573055.351104:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616323200/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573055.351139:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573055.351144:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573055.351146:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573055.351150:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4967 00000100:00080000:0.0:1788573055.351155:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573055.351160:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573055.351171:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573055.351190:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616323328/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573055.351190:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616323200:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788573055.351209:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573055.351213:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573055.351215:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573055.351219:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4967 00000100:00080000:0.0:1788573055.351223:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573055.351228:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573055.351233:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573055.351247:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616323456/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573055.351278:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573055.351316:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573055/real 1788573055] req@ffff9ada4557f800 x1875449616323200/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573080 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573055.351336:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616323200/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573080 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573055.351348:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573055.351369:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573055.351374:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616323200:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573055.351397:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616323328:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573055.351421:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573055.351437:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573055/real 1788573055] req@ffff9ada4557d880 x1875449616323328/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573080 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573055.351450:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616323328/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573080 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573055.351459:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573055.351463:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573055.351466:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616323328:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573055.351473:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616323456:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573055.351494:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573055.351520:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573055/real 1788573055] req@ffff9ada4557fb80 x1875449616323456/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573080 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573055.351532:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616323456/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573080 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573055.351540:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573055.351544:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573055.351547:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616323456:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573055.543023:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573055.863099:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573056.566959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573056.886981:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573057.590936:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573057.911105:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573058.614965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573058.934983:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573059.638951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573059.959007:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573060.471000:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573060.471008:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573060.471012:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573060.471020:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4973 00000100:00080000:3.0:1788573060.471029:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573060.471035:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573060.471074:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0380 x1875449616323584/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573060.471095:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573060.471097:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573060.471099:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573060.471102:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4973 00000100:00080000:3.0:1788573060.471104:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573060.471106:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573060.471114:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573060.471123:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0700 x1875449616323712/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573060.471135:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573060.471138:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573060.471140:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573060.471142:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4973 00000100:00080000:3.0:1788573060.471145:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573060.471147:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573060.471150:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573060.471159:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3480 x1875449616323840/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573060.471161:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616323584:192.168.202.113@tcp:250:kworker.0 00000800:00000100:1.0:1788573060.471229:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573060.471266:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573060/real 1788573060] req@ffff9ada604e0380 x1875449616323584/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573085 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573060.471287:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0380 x1875449616323584/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573085 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573060.471307:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573060.471311:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573060.471316:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616323584:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573060.471324:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616323712:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573060.471348:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573060.471362:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573060/real 1788573060] req@ffff9ada604e0700 x1875449616323712/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573085 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573060.471376:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0700 x1875449616323712/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573085 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573060.471385:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573060.471388:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573060.471392:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616323712:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573060.471417:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616323840:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573060.471440:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573060.471452:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573060/real 1788573060] req@ffff9ada604e3480 x1875449616323840/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573085 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573060.471466:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3480 x1875449616323840/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573085 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573060.471474:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573060.471478:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573060.471481:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616323840:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573060.662945:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573060.982985:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573061.687077:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573062.007011:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573062.710972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573063.031002:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573063.734933:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573064.055006:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573064.758972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573065.079041:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573065.590968:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573065.590976:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573065.590980:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573065.590988:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4978 00000100:00080000:3.0:1788573065.590997:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573065.591003:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573065.591038:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3480 x1875449616323968/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573065.591061:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573065.591063:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573065.591065:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573065.591067:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4978 00000100:00080000:3.0:1788573065.591070:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573065.591072:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573065.591079:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573065.591087:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0700 x1875449616324096/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573065.591094:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573065.591097:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573065.591099:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573065.591102:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4978 00000100:00080000:3.0:1788573065.591103:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573065.591107:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573065.591110:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573065.591119:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0380 x1875449616324224/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573065.591121:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616323968:192.168.202.113@tcp:250:kworker.0 00000800:00000100:1.0:1788573065.591192:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573065.591230:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573065/real 1788573065] req@ffff9ada604e3480 x1875449616323968/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573090 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573065.591249:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3480 x1875449616323968/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573090 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573065.591269:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573065.591274:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573065.591278:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616323968:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573065.591287:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616324096:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573065.591312:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573065.591327:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573065/real 1788573065] req@ffff9ada604e0700 x1875449616324096/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573090 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573065.591340:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0700 x1875449616324096/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573090 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573065.591349:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573065.591353:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573065.591357:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616324096:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573065.591381:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616324224:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573065.591403:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573065.591415:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573065/real 1788573065] req@ffff9ada604e0380 x1875449616324224/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573090 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573065.591446:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0380 x1875449616324224/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573090 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573065.591456:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573065.591460:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573065.591463:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616324224:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573065.782920:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573066.103042:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573066.806970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573067.127104:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573067.830989:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573068.151069:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573068.854976:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573069.175155:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573069.878980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573070.198998:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573070.711054:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573070.711064:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573070.711070:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573070.711081:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4983 00000100:00080000:2.0:1788573070.711094:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573070.711101:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573070.711148:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616324352/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573070.711178:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573070.711182:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573070.711183:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573070.711186:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4983 00000100:00080000:2.0:1788573070.711189:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573070.711192:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573070.711203:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573070.711212:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616324480/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573070.711220:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573070.711223:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573070.711224:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573070.711227:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4983 00000100:00080000:2.0:1788573070.711229:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573070.711232:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573070.711235:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573070.711244:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616324608/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573070.711251:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616324352:192.168.202.113@tcp:250:kworker.0 00000800:00000100:1.0:1788573070.711331:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573070.711374:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573070/real 1788573070] req@ffff9ada60625f80 x1875449616324352/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573095 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573070.711389:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616324352/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573095 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573070.711398:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573070.711401:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573070.711404:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616324352:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573070.711410:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616324480:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573070.711429:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573070.711438:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573070/real 1788573070] req@ffff9ada60624e00 x1875449616324480/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573095 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573070.711446:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616324480/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573095 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573070.711451:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573070.711453:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573070.711455:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616324480:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573070.711484:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616324608:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573070.711497:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573070.711504:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573070/real 1788573070] req@ffff9ada60627480 x1875449616324608/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573095 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573070.711512:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616324608/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573095 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573070.711517:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573070.711519:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573070.711521:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616324608:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573070.902935:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573071.223101:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573071.926952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573072.246988:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573072.950922:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573073.271021:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573073.974933:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573074.295032:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573074.998935:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573075.319058:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573075.830949:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573075.830961:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573075.830965:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573075.830976:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4988 00000100:00080000:0.0:1788573075.830987:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573075.830995:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573075.831036:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616324736/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573075.831061:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573075.831065:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573075.831067:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573075.831069:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4988 00000100:00080000:0.0:1788573075.831072:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573075.831074:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573075.831084:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573075.831093:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616324864/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573075.831101:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573075.831104:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573075.831105:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573075.831119:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4988 00000100:00080000:0.0:1788573075.831122:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573075.831125:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788573075.831131:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616324736:192.168.202.113@tcp:250:kworker.0 00000008:00100000:0.0:1788573075.831132:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573075.831146:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616324992/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573075.831223:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573075.831264:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573075/real 1788573075] req@ffff9ada4557dc00 x1875449616324736/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573100 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573075.831282:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616324736/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573100 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573075.831291:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573075.831294:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573075.831297:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616324736:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573075.831303:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616324864:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573075.831319:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573075.831327:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573075/real 1788573075] req@ffff9ada4557e300 x1875449616324864/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573100 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573075.831336:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616324864/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573100 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573075.831341:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573075.831343:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573075.831345:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616324864:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573075.831368:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616324992:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573075.831380:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573075.831386:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573075/real 1788573075] req@ffff9ada4557e680 x1875449616324992/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573100 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573075.831394:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616324992/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573100 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573075.831405:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573075.831407:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573075.831409:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616324992:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573076.022971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573076.342990:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573077.046947:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573077.367020:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573078.071157:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573078.390987:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573079.094976:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573079.415076:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573080.118977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573080.439019:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573080.950935:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573080.950943:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573080.950946:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573080.950955:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4993 00000100:00080000:0.0:1788573080.950964:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573080.950971:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573080.951012:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d180 x1875449616325120/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573080.951035:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573080.951038:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573080.951040:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573080.951043:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4993 00000100:00080000:0.0:1788573080.951045:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573080.951047:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573080.951054:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573080.951064:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616325248/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573080.951076:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573080.951080:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573080.951082:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573080.951086:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4993 00000100:00080000:0.0:1788573080.951090:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573080.951098:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788573080.951099:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616325120:192.168.202.113@tcp:250:kworker.0 00000008:00100000:0.0:1788573080.951104:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573080.951120:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557df80 x1875449616325376/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573080.951165:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573080.951196:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573080/real 1788573080] req@ffff9ada4557d180 x1875449616325120/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573105 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573080.951219:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d180 x1875449616325120/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573105 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573080.951227:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573080.951230:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573080.951234:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616325120:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573080.951239:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616325248:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573080.951254:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573080.951262:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573080/real 1788573080] req@ffff9ada4557ed80 x1875449616325248/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573105 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573080.951275:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616325248/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573105 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573080.951285:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573080.951287:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573080.951289:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616325248:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573080.951309:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616325376:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573080.951320:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573080.951326:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573080/real 1788573080] req@ffff9ada4557df80 x1875449616325376/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573105 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573080.951333:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557df80 x1875449616325376/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573105 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573080.951339:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573080.951340:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573080.951342:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616325376:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573081.142995:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573081.463053:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573082.167003:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573082.487129:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573083.190934:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573083.511081:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573084.214961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573084.535081:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573085.238940:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573085.559022:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573086.071024:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573086.071038:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573086.071044:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573086.071058:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 4998 00000100:00080000:2.0:1788573086.071072:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573086.071082:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573086.071131:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616325504/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573086.071166:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573086.071170:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573086.071173:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573086.071177:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4998 00000100:00080000:2.0:1788573086.071182:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573086.071186:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573086.071198:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573086.071222:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616325632/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573086.071235:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573086.071240:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573086.071246:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1788573086.071247:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616325504:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788573086.071253:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 4998 00000100:00080000:2.0:1788573086.071263:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573086.071269:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573086.071275:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573086.071289:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616325760/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573086.071339:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573086.071381:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573086/real 1788573086] req@ffff9ada60625180 x1875449616325504/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573111 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573086.071403:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616325504/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573111 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573086.071417:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573086.071422:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573086.071427:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616325504:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573086.071455:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616325632:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573086.071482:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573086.071496:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573086/real 1788573086] req@ffff9ada60626d80 x1875449616325632/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573111 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573086.071511:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616325632/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573111 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573086.071521:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573086.071525:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573086.071529:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616325632:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573086.071537:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616325760:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573086.071561:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573086.071574:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573086/real 1788573086] req@ffff9ada60627b80 x1875449616325760/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573111 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573086.071587:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616325760/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573111 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573086.071596:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573086.071600:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573086.071603:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616325760:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573086.262982:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573086.583122:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573087.287003:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573087.607050:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573088.310980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573088.631057:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573089.334959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573089.654977:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573090.358937:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573090.679082:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573091.190952:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573091.190964:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573091.190968:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573091.190979:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5003 00000100:00080000:2.0:1788573091.190989:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573091.190997:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573091.191046:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626300 x1875449616325888/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573091.191087:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573091.191092:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573091.191094:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573091.191098:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5003 00000100:00080000:2.0:1788573091.191102:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573091.191106:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573091.191116:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573091.191131:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627800 x1875449616326016/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573091.191135:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616325888:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788573091.191152:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573091.191156:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573091.191159:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573091.191198:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5003 00000100:00080000:2.0:1788573091.191204:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573091.191208:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573091.191215:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000800:00000100:1.0:1788573091.191227:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788573091.191232:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616326144/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573091.191263:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573091/real 1788573091] req@ffff9ada60626300 x1875449616325888/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573116 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573091.191278:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626300 x1875449616325888/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573116 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573091.191286:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573091.191289:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573091.191292:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616325888:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573091.191311:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616326016:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573091.191325:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573091.191333:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573091/real 1788573091] req@ffff9ada60627800 x1875449616326016/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573116 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573091.191341:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627800 x1875449616326016/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573116 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573091.191346:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573091.191348:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573091.191350:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616326016:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573091.191354:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616326144:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573091.191364:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573091.191370:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573091/real 1788573091] req@ffff9ada60627b80 x1875449616326144/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573116 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573091.191378:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616326144/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573116 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573091.191383:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573091.191384:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573091.191386:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616326144:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573091.382961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573091.703047:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573092.406941:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573092.727129:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573093.430991:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573093.751287:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573094.454990:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573094.775062:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573095.478971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573095.799045:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573096.310926:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573096.310940:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573096.310945:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573096.310957:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5008 00000100:00080000:2.0:1788573096.310972:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573096.310981:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573096.311032:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616326272/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573096.311062:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573096.311065:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573096.311067:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573096.311070:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5008 00000100:00080000:2.0:1788573096.311072:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573096.311075:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573096.311085:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573096.311095:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616326400/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573096.311103:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573096.311106:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573096.311108:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573096.311111:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5008 00000100:00080000:2.0:1788573096.311113:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573096.311115:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573096.311119:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573096.311123:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616326272:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788573096.311130:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616326528/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573096.311205:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573096.311246:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573096/real 1788573096] req@ffff9ada60626d80 x1875449616326272/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573121 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573096.311263:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616326272/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573121 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573096.311272:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573096.311276:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573096.311279:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616326272:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573096.311286:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616326400:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573096.311302:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573096.311309:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573096/real 1788573096] req@ffff9ada60625180 x1875449616326400/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573121 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573096.311318:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616326400/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573121 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573096.311341:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573096.311344:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573096.311346:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616326400:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573096.311374:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616326528:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573096.311389:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573096.311397:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573096/real 1788573096] req@ffff9ada60627480 x1875449616326528/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573121 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573096.311405:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616326528/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573121 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573096.311411:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573096.311413:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573096.311415:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616326528:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573096.502963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573096.823035:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573097.526929:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573097.846994:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573098.550941:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573098.871093:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573099.574998:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573099.894993:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573100.598938:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573100.919023:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788573101.431007:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573101.431018:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788573101.431022:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573101.431032:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5013 00000100:00080000:1.0:1788573101.431042:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573101.431050:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788573101.431092:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e680 x1875449616326656/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573101.431123:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573101.431143:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788573101.431145:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573101.431149:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5013 00000100:00080000:1.0:1788573101.431153:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573101.431157:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573101.431167:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573101.431182:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d880 x1875449616326784/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573101.431192:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573101.431195:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788573101.431198:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573101.431203:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5013 00000100:00080000:1.0:1788573101.431207:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573101.431213:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573101.431222:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616326656:192.168.202.113@tcp:250:kworker.0 00000008:00100000:1.0:1788573101.431224:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573101.431248:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ce00 x1875449616326912/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573101.431318:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573101.431361:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573101/real 1788573101] req@ffff9ada4566e680 x1875449616326656/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573126 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573101.431380:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e680 x1875449616326656/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573126 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573101.431389:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573101.431393:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573101.431396:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616326656:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573101.431403:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616326784:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573101.431421:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573101.431429:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573101/real 1788573101] req@ffff9ada4566d880 x1875449616326784/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573126 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573101.431442:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d880 x1875449616326784/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573126 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573101.431448:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573101.431450:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573101.431452:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616326784:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573101.431519:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616326912:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573101.431535:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573101.431544:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573101/real 1788573101] req@ffff9ada4566ce00 x1875449616326912/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573126 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573101.431553:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ce00 x1875449616326912/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573126 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573101.431558:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573101.431561:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573101.431563:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616326912:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573101.623003:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573101.942968:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573102.646944:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573102.967061:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573103.670957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573103.991086:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573104.694936:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573105.015124:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573105.718937:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573106.039025:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788573106.550973:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573106.550984:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788573106.550989:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573106.551000:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5019 00000100:00080000:1.0:1788573106.551011:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573106.551020:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788573106.551061:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c700 x1875449616327040/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573106.551093:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573106.551097:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788573106.551099:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573106.551103:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5019 00000100:00080000:1.0:1788573106.551107:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573106.551111:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573106.551120:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573106.551134:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c380 x1875449616327168/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573106.551146:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573106.551152:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788573106.551154:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573106.551158:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5019 00000100:00080000:1.0:1788573106.551162:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573106.551166:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573106.551171:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573106.551181:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616327040:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573106.551190:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f100 x1875449616327296/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573106.551266:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573106.551309:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573106/real 1788573106] req@ffff9ada4566c700 x1875449616327040/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573131 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573106.551335:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c700 x1875449616327040/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573131 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573106.551346:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573106.551350:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573106.551353:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616327040:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573106.551359:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616327168:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573106.551375:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573106.551384:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573106/real 1788573106] req@ffff9ada4566c380 x1875449616327168/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573131 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573106.551392:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c380 x1875449616327168/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573131 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573106.551398:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573106.551400:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573106.551402:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616327168:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573106.551425:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616327296:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573106.551436:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573106.551442:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573106/real 1788573106] req@ffff9ada4566f100 x1875449616327296/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573131 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573106.551450:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f100 x1875449616327296/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573131 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573106.551460:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573106.551463:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573106.551465:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616327296:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573106.742972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573107.063028:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573107.766984:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573108.087108:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573108.790997:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573109.111009:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573109.814919:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573110.135063:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573110.839065:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573111.159021:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573111.670986:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573111.670999:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573111.671004:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573111.671016:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5024 00000100:00080000:2.0:1788573111.671026:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573111.671035:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573111.671075:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616327424/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573111.671115:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573111.671119:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573111.671121:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573111.671125:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5024 00000100:00080000:2.0:1788573111.671129:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573111.671134:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573111.671144:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573111.671158:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616327552/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788573111.671168:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616327424:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788573111.671172:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573111.671178:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573111.671180:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573111.671185:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5024 00000100:00080000:2.0:1788573111.671189:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573111.671194:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573111.671200:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573111.671212:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616327680/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573111.671229:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573111.671268:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573111/real 1788573111] req@ffff9ada60624e00 x1875449616327424/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573136 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573111.671286:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616327424/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573136 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573111.671294:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573111.671297:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573111.671300:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616327424:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573111.671319:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616327552:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573111.671337:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573111.671346:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573111/real 1788573111] req@ffff9ada60625f80 x1875449616327552/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573136 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573111.671369:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616327552/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573136 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573111.671375:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573111.671377:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573111.671379:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616327552:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573111.671384:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616327680:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573111.671396:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573111.671403:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573111/real 1788573111] req@ffff9ada60625c00 x1875449616327680/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573136 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573111.671411:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616327680/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573136 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573111.671416:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573111.671418:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573111.671420:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616327680:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573111.862913:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573112.183013:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573112.886969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573113.207009:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573113.910954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573114.231028:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573114.934957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573115.255025:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573115.958959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573116.279018:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788573116.791100:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573116.791115:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788573116.791121:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573116.791134:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5029 00000100:00080000:1.0:1788573116.791147:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573116.791156:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788573116.791210:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ca80 x1875449616327808/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573116.791238:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573116.791242:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788573116.791243:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573116.791246:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5029 00000100:00080000:1.0:1788573116.791249:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573116.791252:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573116.791264:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573116.791275:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d500 x1875449616327936/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573116.791283:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573116.791287:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788573116.791290:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1788573116.791291:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616327808:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788573116.791294:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5029 00000100:00080000:1.0:1788573116.791298:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573116.791304:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573116.791310:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573116.791317:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d180 x1875449616328064/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788573116.791421:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616327936:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573116.791451:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616328064:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573116.982952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573117.303017:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573118.006961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573118.327072:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573119.031031:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573119.351026:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1788573119.863122:0:2370:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:2.0:1788573119.863159:0:2370:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1788573119.863174:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788573119.863216:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788573119.863227:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:3.0:1788573119.863294:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573116/real 1788573119] req@ffff9ada4566d500 x1875449616327936/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573141 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573119.863320:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d500 x1875449616327936/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573141 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573119.863333:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573119.863339:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573119.863343:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616327936:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573119.863352:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573116/real 1788573119] req@ffff9ada4566d180 x1875449616328064/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573141 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573119.863365:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d180 x1875449616328064/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573141 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573119.863388:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573119.863393:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573119.863397:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616328064:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573119.863404:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573116/real 1788573119] req@ffff9ada4566ca80 x1875449616327808/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573141 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573119.863450:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ca80 x1875449616327808/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573141 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573119.863458:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573119.863461:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573119.863464:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616327808:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573120.054936:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573120.375031:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573121.078963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573121.398955:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788573121.910964:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573121.910972:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788573121.910976:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573121.910985:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5034 00000100:00080000:1.0:1788573121.910994:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573121.911000:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788573121.911031:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f480 x1875449616328192/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573121.911056:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573121.911060:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788573121.911062:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573121.911066:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5034 00000100:00080000:1.0:1788573121.911070:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573121.911074:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573121.911084:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573121.911098:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e300 x1875449616328320/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573121.911110:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573121.911112:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788573121.911114:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573121.911117:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5034 00000100:00080000:1.0:1788573121.911120:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573121.911125:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573121.911125:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616328192:192.168.202.113@tcp:250:kworker.0 00000008:00100000:1.0:1788573121.911135:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573121.911149:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ea00 x1875449616328448/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573121.911195:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573121.911228:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573121/real 1788573121] req@ffff9ada4566f480 x1875449616328192/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573146 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573121.911249:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f480 x1875449616328192/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573146 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573121.911261:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573121.911266:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573121.911270:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616328192:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573121.911294:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616328320:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573121.911318:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573121.911331:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573121/real 1788573121] req@ffff9ada4566e300 x1875449616328320/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573146 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573121.911377:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e300 x1875449616328320/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573146 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573121.911387:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573121.911390:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573121.911409:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616328320:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573121.911416:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616328448:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573121.911437:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573121.911450:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573121/real 1788573121] req@ffff9ada4566ea00 x1875449616328448/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573146 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573121.911469:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ea00 x1875449616328448/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573146 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573121.911479:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573121.911483:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573121.911486:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616328448:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573122.102940:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573122.423008:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573123.126962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573123.447032:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573124.150972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573124.471029:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573125.174950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573125.495035:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573126.198973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573126.519001:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573127.030975:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573127.030984:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573127.030989:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573127.031000:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5039 00000100:00080000:3.0:1788573127.031010:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573127.031018:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573127.031054:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3800 x1875449616328576/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573127.031089:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573127.031093:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573127.031095:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573127.031100:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5039 00000100:00080000:3.0:1788573127.031104:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573127.031111:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788573127.031112:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616328576:192.168.202.113@tcp:250:kworker.0 00000008:00100000:3.0:1788573127.031121:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573127.031136:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1880 x1875449616328704/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573127.031159:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573127.031163:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573127.031165:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573127.031169:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5039 00000100:00080000:3.0:1788573127.031174:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573127.031178:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000800:00000100:1.0:1788573127.031183:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000008:00100000:3.0:1788573127.031184:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573127.031199:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1180 x1875449616328832/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573127.031220:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573127/real 1788573127] req@ffff9ada604e3800 x1875449616328576/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573152 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573127.031235:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3800 x1875449616328576/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573152 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573127.031244:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573127.031247:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573127.031250:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616328576:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573127.031271:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616328704:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573127.031287:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573127.031295:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573127/real 1788573127] req@ffff9ada604e1880 x1875449616328704/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573152 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573127.031303:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1880 x1875449616328704/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573152 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573127.031309:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573127.031311:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573127.031313:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616328704:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573127.031317:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616328832:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573127.031327:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573127.031333:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573127/real 1788573127] req@ffff9ada604e1180 x1875449616328832/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573152 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573127.031340:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1180 x1875449616328832/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573152 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573127.031346:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573127.031347:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573127.031349:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616328832:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573127.222948:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573127.542974:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573128.246995:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573128.567003:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573129.270938:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573129.591117:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573130.294976:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573130.615039:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573131.319007:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573131.639079:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573132.150989:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573132.150997:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573132.151001:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573132.151010:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5044 00000100:00080000:3.0:1788573132.151020:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573132.151027:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573132.151061:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2300 x1875449616328960/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573132.151083:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573132.151086:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573132.151088:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573132.151090:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5044 00000100:00080000:3.0:1788573132.151093:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573132.151095:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573132.151102:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573132.151110:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2d80 x1875449616329088/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573132.151118:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573132.151120:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573132.151122:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573132.151124:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5044 00000100:00080000:3.0:1788573132.151126:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573132.151128:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573132.151133:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573132.151139:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0a80 x1875449616329216/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573132.151144:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616328960:192.168.202.113@tcp:250:kworker.0 00000800:00000100:1.0:1788573132.151208:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573132.151244:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573132/real 1788573132] req@ffff9ada604e2300 x1875449616328960/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573157 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573132.151269:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2300 x1875449616328960/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573157 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573132.151280:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573132.151285:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573132.151289:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616328960:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573132.151300:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616329088:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573132.151322:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573132.151335:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573132/real 1788573132] req@ffff9ada604e2d80 x1875449616329088/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573157 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573132.151348:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2d80 x1875449616329088/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573157 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573132.151356:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573132.151359:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573132.151362:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616329088:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573132.151387:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616329216:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573132.151404:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573132.151412:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573132/real 1788573132] req@ffff9ada604e0a80 x1875449616329216/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573157 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573132.151420:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0a80 x1875449616329216/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573157 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573132.151430:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573132.151432:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573132.151434:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616329216:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573132.342970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573132.663080:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573133.366937:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573133.687033:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573134.390952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573134.711056:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573135.414946:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573135.735028:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573136.438966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573136.759026:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788573137.270979:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573137.270990:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788573137.270995:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573137.271007:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5049 00000100:00080000:1.0:1788573137.271020:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573137.271027:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788573137.271076:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ed80 x1875449616329344/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573137.271112:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573137.271117:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788573137.271119:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573137.271124:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5049 00000100:00080000:1.0:1788573137.271129:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573137.271134:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573137.271146:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573137.271154:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616329344:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573137.271161:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f800 x1875449616329472/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573137.271171:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573137.271173:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788573137.271175:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573137.271179:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5049 00000100:00080000:1.0:1788573137.271182:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573137.271185:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573137.271188:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573137.271198:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566dc00 x1875449616329600/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573137.271242:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573137.271280:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573137/real 1788573137] req@ffff9ada4566ed80 x1875449616329344/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573162 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573137.271297:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ed80 x1875449616329344/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573162 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573137.271305:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573137.271308:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573137.271311:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616329344:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573137.271335:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616329472:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573137.271359:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573137.271368:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573137/real 1788573137] req@ffff9ada4566f800 x1875449616329472/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573162 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573137.271376:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f800 x1875449616329472/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573162 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573137.271388:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573137.271390:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573137.271393:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616329472:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573137.271398:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616329600:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573137.271419:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573137.271432:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573137/real 1788573137] req@ffff9ada4566dc00 x1875449616329600/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573162 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573137.271446:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566dc00 x1875449616329600/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573162 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573137.271455:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573137.271459:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573137.271462:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616329600:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573137.462963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573137.783055:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573138.486960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573138.806979:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573139.510941:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573139.831080:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573140.534961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573140.854998:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573141.558938:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573141.879002:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573142.390957:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573142.390967:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573142.390971:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573142.390981:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5054 00000100:00080000:2.0:1788573142.390990:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573142.390996:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573142.391028:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616329728/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573142.391049:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573142.391053:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573142.391055:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573142.391059:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5054 00000100:00080000:2.0:1788573142.391063:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573142.391066:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573142.391077:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573142.391090:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616329856/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788573142.391092:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616329728:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788573142.391101:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573142.391116:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573142.391118:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573142.391122:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5054 00000100:00080000:2.0:1788573142.391126:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573142.391129:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573142.391134:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573142.391143:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616329984/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573142.391152:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573142.391185:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573142/real 1788573142] req@ffff9ada60625880 x1875449616329728/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573167 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573142.391211:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616329728/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573167 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573142.391219:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573142.391222:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573142.391225:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616329728:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573142.391245:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616329856:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573142.391270:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573142.391279:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573142/real 1788573142] req@ffff9ada60624a80 x1875449616329856/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573167 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573142.391288:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616329856/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573167 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573142.391294:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573142.391296:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573142.391298:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616329856:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573142.391302:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616329984:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573142.391313:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573142.391318:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573142/real 1788573142] req@ffff9ada60626a00 x1875449616329984/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573167 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573142.391326:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616329984/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573167 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573142.391331:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573142.391333:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573142.391335:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616329984:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573142.582948:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573142.903089:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573143.606972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573143.927110:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573144.630948:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573144.950989:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573145.654919:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573145.975033:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573146.678967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573146.999006:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573147.510925:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573147.510933:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573147.510937:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573147.510945:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5060 00000100:00080000:2.0:1788573147.510953:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573147.510959:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573147.510991:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616330112/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573147.511021:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573147.511025:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573147.511026:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573147.511029:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5060 00000100:00080000:2.0:1788573147.511031:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573147.511034:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573147.511044:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573147.511054:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616330240/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573147.511063:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573147.511067:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1788573147.511068:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616330112:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788573147.511069:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573147.511077:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5060 00000100:00080000:2.0:1788573147.511080:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573147.511082:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573147.511085:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573147.511095:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616330368/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573147.511129:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573147.511166:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573147/real 1788573147] req@ffff9ada60624380 x1875449616330112/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573172 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573147.511187:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616330112/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573172 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573147.511195:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573147.511198:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573147.511201:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616330112:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573147.511220:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616330240:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573147.511247:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573147.511256:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573147/real 1788573147] req@ffff9ada60625500 x1875449616330240/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573172 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573147.511265:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616330240/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573172 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573147.511271:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573147.511273:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573147.511275:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616330240:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573147.511280:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616330368:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573147.511291:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573147.511297:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573147/real 1788573147] req@ffff9ada60626680 x1875449616330368/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573172 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573147.511310:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616330368/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573172 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573147.511318:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573147.511321:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573147.511325:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616330368:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573147.702972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573148.023020:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573148.726950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573149.046981:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573149.750993:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573150.070996:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573150.774947:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573151.095074:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573151.798978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573152.119052:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788573152.631030:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573152.631045:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788573152.631051:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573152.631066:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5065 00000100:00080000:1.0:1788573152.631080:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573152.631089:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788573152.631138:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c000 x1875449616330496/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573152.631176:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573152.631179:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788573152.631181:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573152.631184:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5065 00000100:00080000:1.0:1788573152.631187:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573152.631190:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573152.631200:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573152.631211:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566df80 x1875449616330624/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573152.631249:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573152.631252:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788573152.631254:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1788573152.631262:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616330496:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788573152.631263:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5065 00000100:00080000:1.0:1788573152.631274:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573152.631280:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573152.631285:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573152.631302:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566fb80 x1875449616330752/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573152.631343:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573152.631379:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573152/real 1788573152] req@ffff9ada4566c000 x1875449616330496/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573177 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573152.631396:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c000 x1875449616330496/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573177 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573152.631407:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573152.631411:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573152.631428:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616330496:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573152.631460:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616330624:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573152.631489:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573152.631501:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573152/real 1788573152] req@ffff9ada4566df80 x1875449616330624/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573177 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573152.631512:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566df80 x1875449616330624/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573177 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573152.631519:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573152.631522:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573152.631525:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616330624:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573152.631530:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616330752:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573152.631546:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573152.631553:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573152/real 1788573152] req@ffff9ada4566fb80 x1875449616330752/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573177 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573152.631562:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566fb80 x1875449616330752/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573177 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573152.631568:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573152.631570:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573152.631573:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616330752:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573152.822954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573153.143111:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573153.846965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573154.166978:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573154.870961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573155.191089:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573155.894961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573156.215039:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573156.918948:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573157.239037:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573157.750991:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573157.750999:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573157.751003:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573157.751012:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5070 00000100:00080000:3.0:1788573157.751021:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573157.751027:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573157.751079:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0000 x1875449616330880/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573157.751103:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573157.751106:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573157.751108:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573157.751111:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5070 00000100:00080000:3.0:1788573157.751113:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573157.751116:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573157.751125:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573157.751138:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1c00 x1875449616331008/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573157.751139:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616330880:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788573157.751153:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573157.751156:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573157.751157:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573157.751160:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5070 00000100:00080000:3.0:1788573157.751162:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573157.751165:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573157.751169:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573157.751179:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3b80 x1875449616331136/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573157.751203:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573157.751243:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573157/real 1788573157] req@ffff9ada604e0000 x1875449616330880/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573182 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573157.751261:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0000 x1875449616330880/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573182 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573157.751269:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573157.751271:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573157.751274:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616330880:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573157.751293:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616331008:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573157.751308:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573157.751316:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573157/real 1788573157] req@ffff9ada604e1c00 x1875449616331008/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573182 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573157.751325:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1c00 x1875449616331008/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573182 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573157.751330:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573157.751333:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573157.751335:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616331008:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573157.751338:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616331136:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573157.751348:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573157.751354:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573157/real 1788573157] req@ffff9ada604e3b80 x1875449616331136/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573182 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573157.751366:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3b80 x1875449616331136/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573182 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573157.751374:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573157.751378:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573157.751381:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616331136:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573157.942949:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573158.263119:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573158.966941:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573159.287041:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573159.990951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573160.311007:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573161.014950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573161.335018:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573162.039097:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573162.359080:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573162.870989:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573162.871000:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573162.871005:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573162.871017:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5075 00000100:00080000:3.0:1788573162.871028:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573162.871038:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573162.871075:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3100 x1875449616331264/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573162.871107:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573162.871116:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573162.871118:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573162.871123:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5075 00000100:00080000:3.0:1788573162.871127:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573162.871132:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573162.871142:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573162.871157:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616331264:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573162.871163:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1f80 x1875449616331392/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573162.871185:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573162.871189:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573162.871192:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573162.871196:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5075 00000100:00080000:3.0:1788573162.871201:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573162.871206:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573162.871211:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000800:00000100:1.0:1788573162.871218:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573162.871237:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2680 x1875449616331520/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573162.871251:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573162/real 1788573162] req@ffff9ada604e3100 x1875449616331264/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573187 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573162.871267:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3100 x1875449616331264/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573187 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573162.871280:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573162.871283:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573162.871287:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616331264:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573162.871309:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616331392:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573162.871331:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573162.871345:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573162/real 1788573162] req@ffff9ada604e1f80 x1875449616331392/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573187 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573162.871357:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1f80 x1875449616331392/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573187 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573162.871366:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573162.871369:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573162.871372:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616331392:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573162.871378:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616331520:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573162.871398:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573162.871410:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573162/real 1788573162] req@ffff9ada604e2680 x1875449616331520/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573187 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573162.871423:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2680 x1875449616331520/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573187 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573162.871429:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573162.871431:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573162.871433:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616331520:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573163.062946:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573163.383095:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573164.086986:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573164.407066:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573165.110973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573165.431045:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573166.134969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573166.455077:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573167.158965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573167.479090:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573167.990971:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573167.990980:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573167.990983:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573167.990992:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5080 00000100:00080000:3.0:1788573167.991002:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573167.991007:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573167.991042:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1500 x1875449616331648/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573167.991067:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573167.991070:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573167.991071:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573167.991074:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5080 00000100:00080000:3.0:1788573167.991077:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573167.991079:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573167.991087:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573167.991095:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0e00 x1875449616331776/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573167.991102:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573167.991105:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573167.991106:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573167.991109:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5080 00000100:00080000:3.0:1788573167.991111:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573167.991114:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573167.991117:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573167.991123:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2a00 x1875449616331904/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573167.991131:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616331648:192.168.202.113@tcp:250:kworker.0 00000800:00000100:1.0:1788573167.991194:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573167.991228:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573167/real 1788573167] req@ffff9ada604e1500 x1875449616331648/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573192 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573167.991246:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1500 x1875449616331648/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573192 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573167.991258:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573167.991263:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573167.991266:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616331648:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573167.991274:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616331776:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573167.991295:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573167.991307:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573167/real 1788573167] req@ffff9ada604e0e00 x1875449616331776/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573192 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573167.991319:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0e00 x1875449616331776/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573192 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573167.991328:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573167.991331:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573167.991335:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616331776:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573167.991360:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616331904:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573167.991383:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573167.991395:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573167/real 1788573167] req@ffff9ada604e2a00 x1875449616331904/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573192 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573167.991407:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2a00 x1875449616331904/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573192 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573167.991415:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573167.991419:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573167.991422:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616331904:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573168.182967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573168.503047:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573169.206986:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573169.527048:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573170.230955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573170.551084:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573171.254987:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573171.575070:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573172.278926:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573172.599065:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573173.110976:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573173.110984:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573173.110987:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573173.110996:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5085 00000100:00080000:0.0:1788573173.111005:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573173.111012:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573173.111052:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616332032/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573173.111073:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573173.111076:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573173.111078:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573173.111081:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5085 00000100:00080000:0.0:1788573173.111083:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573173.111086:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573173.111094:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573173.111103:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d500 x1875449616332160/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573173.111110:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573173.111113:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573173.111114:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573173.111117:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5085 00000100:00080000:0.0:1788573173.111119:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573173.111121:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573173.111124:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573173.111126:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616332032:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788573173.111140:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557df80 x1875449616332288/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573173.111182:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573173.111212:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573173/real 1788573173] req@ffff9ada4557ea00 x1875449616332032/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573198 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573173.111225:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616332032/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573198 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573173.111232:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573173.111235:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573173.111238:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616332032:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573173.111243:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616332160:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573173.111259:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573173.111267:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573173/real 1788573173] req@ffff9ada4557d500 x1875449616332160/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573198 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573173.111275:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d500 x1875449616332160/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573198 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573173.111280:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573173.111283:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573173.111285:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616332160:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573173.111303:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616332288:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573173.111313:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573173.111319:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573173/real 1788573173] req@ffff9ada4557df80 x1875449616332288/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573198 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573173.111327:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557df80 x1875449616332288/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573198 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573173.111332:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573173.111334:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573173.111336:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616332288:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573173.302949:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573173.623040:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573174.326950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573174.647040:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573175.350934:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573175.671022:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573176.374952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573176.695037:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573177.398956:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573177.719014:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573178.231076:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573178.231091:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573178.231097:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573178.231110:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5090 00000100:00080000:3.0:1788573178.231125:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573178.231135:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573178.231188:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2a00 x1875449616332416/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573178.231220:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573178.231240:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573178.231242:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573178.231247:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5090 00000100:00080000:3.0:1788573178.231251:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573178.231256:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573178.231269:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573178.231303:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616332416:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573178.231303:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0e00 x1875449616332544/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573178.231321:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573178.231324:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573178.231326:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573178.231358:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5090 00000100:00080000:3.0:1788573178.231362:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573178.231367:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573178.231373:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573178.231386:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1500 x1875449616332672/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573178.231411:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573178.231458:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573178/real 1788573178] req@ffff9ada604e2a00 x1875449616332416/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573203 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573178.231474:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2a00 x1875449616332416/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573203 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573178.231484:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573178.231488:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573178.231491:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616332416:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573178.231516:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616332544:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573178.231533:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573178.231542:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573178/real 1788573178] req@ffff9ada604e0e00 x1875449616332544/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573203 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573178.231551:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0e00 x1875449616332544/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573203 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573178.231557:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573178.231559:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573178.231561:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616332544:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573178.231565:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616332672:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573178.231576:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573178.231582:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573178/real 1788573178] req@ffff9ada604e1500 x1875449616332672/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573203 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573178.231590:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1500 x1875449616332672/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573203 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573178.231600:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573178.231602:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573178.231604:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616332672:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573178.422937:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573178.742996:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573179.446977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573179.767024:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573180.470934:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573180.791022:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573181.494965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573181.815003:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573182.518913:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573182.839009:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573183.350972:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573183.350980:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573183.350983:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573183.350992:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5095 00000100:00080000:0.0:1788573183.351001:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573183.351007:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573183.351040:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616332800/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573183.351067:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573183.351070:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573183.351072:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573183.351074:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5095 00000100:00080000:0.0:1788573183.351076:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573183.351078:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573183.351086:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573183.351095:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d180 x1875449616332928/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573183.351103:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1788573183.351106:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616332800:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788573183.351107:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573183.351112:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573183.351117:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5095 00000100:00080000:0.0:1788573183.351119:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573183.351122:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573183.351126:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573183.351134:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616333056/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573183.351214:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616332928:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573183.351250:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616333056:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573183.542960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573183.863015:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573184.566929:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573184.887007:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573185.590971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573185.911023:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1788573186.423120:0:2372:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:0.0:1788573186.423156:0:2372:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1788573186.423169:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:0.0:1788573186.423214:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:0.0:1788573186.423224:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:1.0:1788573186.423254:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573183/real 1788573186] req@ffff9ada4557d180 x1875449616332928/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573208 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573186.423282:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d180 x1875449616332928/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573208 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573186.423295:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573186.423301:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573186.423306:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616332928:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573186.423315:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573183/real 1788573186] req@ffff9ada4557e680 x1875449616333056/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573208 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573186.423331:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616333056/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573208 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573186.423339:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573186.423343:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573186.423346:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616333056:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573186.423353:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573183/real 1788573186] req@ffff9ada4557ed80 x1875449616332800/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573208 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573186.423366:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616332800/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573208 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573186.423375:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573186.423379:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573186.423383:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616332800:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573186.614968:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573186.935005:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573187.638928:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573187.959054:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573188.470954:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573188.470964:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573188.470968:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573188.471007:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5101 00000100:00080000:3.0:1788573188.471017:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573188.471025:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573188.471062:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2680 x1875449616333184/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573188.471091:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573188.471096:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573188.471098:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573188.471102:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5101 00000100:00080000:3.0:1788573188.471105:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573188.471110:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573188.471121:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573188.471135:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1f80 x1875449616333312/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573188.471142:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616333184:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788573188.471150:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573188.471155:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573188.471157:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573188.471162:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5101 00000100:00080000:3.0:1788573188.471166:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573188.471170:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573188.471174:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573188.471186:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3100 x1875449616333440/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573188.471222:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573188.471257:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573188/real 1788573188] req@ffff9ada604e2680 x1875449616333184/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573213 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573188.471276:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2680 x1875449616333184/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573213 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573188.471288:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573188.471292:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573188.471295:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616333184:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573188.471320:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616333312:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573188.471345:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573188.471357:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573188/real 1788573188] req@ffff9ada604e1f80 x1875449616333312/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573213 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573188.471369:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1f80 x1875449616333312/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573213 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573188.471377:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573188.471380:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573188.471383:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616333312:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573188.471390:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616333440:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573188.471412:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573188.471424:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573188/real 1788573188] req@ffff9ada604e3100 x1875449616333440/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573213 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573188.471436:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3100 x1875449616333440/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573213 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573188.471445:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573188.471449:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573188.471452:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616333440:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573188.662986:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573188.983004:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573189.686959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573190.007011:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573190.710983:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573191.031053:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573191.734941:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573192.055033:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573192.758947:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573193.079007:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573193.590970:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573193.590982:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573193.590986:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573193.590994:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5106 00000100:00080000:3.0:1788573193.591003:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573193.591010:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573193.591043:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3b80 x1875449616333568/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573193.591072:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573193.591075:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573193.591076:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573193.591079:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5106 00000100:00080000:3.0:1788573193.591081:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573193.591084:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573193.591091:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573193.591100:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1c00 x1875449616333696/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573193.591107:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573193.591110:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573193.591111:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573193.591113:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5106 00000100:00080000:3.0:1788573193.591115:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573193.591117:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573193.591122:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573193.591123:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616333568:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573193.591132:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0000 x1875449616333824/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573193.591181:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573193.591212:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573193/real 1788573193] req@ffff9ada604e3b80 x1875449616333568/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573218 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573193.591226:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3b80 x1875449616333568/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573218 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573193.591234:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573193.591237:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573193.591240:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616333568:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573193.591244:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616333696:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573193.591258:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573193.591265:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573193/real 1788573193] req@ffff9ada604e1c00 x1875449616333696/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573218 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573193.591273:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1c00 x1875449616333696/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573218 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573193.591279:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573193.591280:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573193.591282:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616333696:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573193.591300:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616333824:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573193.591311:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573193.591317:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573193/real 1788573193] req@ffff9ada604e0000 x1875449616333824/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573218 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573193.591325:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0000 x1875449616333824/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573218 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573193.591334:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573193.591337:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573193.591339:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616333824:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573193.782949:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573194.103088:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573194.806992:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573195.127086:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573195.830966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573196.151093:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573196.854992:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573197.175052:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573197.878999:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573198.199123:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573198.711002:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573198.711013:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573198.711017:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573198.711028:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5111 00000100:00080000:0.0:1788573198.711038:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573198.711047:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573198.711086:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616333952/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573198.711119:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573198.711123:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573198.711125:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573198.711129:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5111 00000100:00080000:0.0:1788573198.711133:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573198.711138:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573198.711147:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573198.711162:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616334080/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573198.711175:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1788573198.711175:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616333952:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788573198.711184:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573198.711186:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573198.711191:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5111 00000100:00080000:0.0:1788573198.711196:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573198.711200:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573198.711205:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573198.711218:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616334208/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573198.711234:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573198.711268:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573198/real 1788573198] req@ffff9ada4557e300 x1875449616333952/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573223 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573198.711282:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616333952/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573223 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573198.711290:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573198.711293:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573198.711296:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616333952:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573198.711313:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616334080:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573198.711328:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573198.711336:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573198/real 1788573198] req@ffff9ada4557dc00 x1875449616334080/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573223 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573198.711344:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616334080/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573223 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573198.711350:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573198.711352:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573198.711354:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616334080:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573198.711358:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616334208:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573198.711369:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573198.711375:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573198/real 1788573198] req@ffff9ada4557fb80 x1875449616334208/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573223 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573198.711383:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616334208/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573223 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573198.711389:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573198.711392:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573198.711395:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616334208:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573198.902956:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573199.223111:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573199.926933:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573200.247042:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573200.950961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573201.271219:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573201.974952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573202.295051:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573202.998961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573203.319086:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573203.830977:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573203.830990:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573203.830996:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573203.831006:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5116 00000100:00080000:3.0:1788573203.831017:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573203.831024:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573203.831057:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0a80 x1875449616334336/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573203.831080:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573203.831083:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573203.831084:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573203.831088:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5116 00000100:00080000:3.0:1788573203.831091:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573203.831096:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573203.831107:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573203.831122:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616334336:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573203.831125:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2d80 x1875449616334464/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573203.831142:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573203.831145:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573203.831146:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573203.831151:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5116 00000100:00080000:3.0:1788573203.831154:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573203.831158:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573203.831162:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573203.831173:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2300 x1875449616334592/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573203.831184:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573203.831218:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573203/real 1788573203] req@ffff9ada604e0a80 x1875449616334336/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573228 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573203.831232:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0a80 x1875449616334336/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573228 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573203.831251:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573203.831254:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573203.831257:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616334336:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573203.831276:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616334464:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573203.831292:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573203.831300:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573203/real 1788573203] req@ffff9ada604e2d80 x1875449616334464/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573228 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573203.831308:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2d80 x1875449616334464/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573228 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573203.831314:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573203.831316:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573203.831318:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616334464:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573203.831322:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616334592:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573203.831337:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573203.831347:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573203/real 1788573203] req@ffff9ada604e2300 x1875449616334592/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573228 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573203.831359:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2300 x1875449616334592/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573228 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573203.831368:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573203.831372:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573203.831375:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616334592:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573204.022997:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573204.343032:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573205.046990:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573205.367045:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573206.071013:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573206.390990:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573207.094986:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573207.415039:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573208.118972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573208.439064:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788573208.951016:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573208.951031:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788573208.951036:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573208.951051:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5121 00000100:00080000:1.0:1788573208.951066:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573208.951075:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788573208.951122:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566fb80 x1875449616334720/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573208.951153:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573208.951158:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788573208.951160:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573208.951166:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5121 00000100:00080000:1.0:1788573208.951171:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573208.951176:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573208.951189:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573208.951205:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616334720:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573208.951209:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566df80 x1875449616334848/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573208.951230:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573208.951232:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788573208.951235:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573208.951241:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5121 00000100:00080000:1.0:1788573208.951244:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573208.951247:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000800:00000100:3.0:1788573208.951282:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000008:00100000:1.0:1788573208.951302:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573208.951325:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c000 x1875449616334976/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788573208.951326:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573208/real 1788573208] req@ffff9ada4566fb80 x1875449616334720/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573233 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573208.951383:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566fb80 x1875449616334720/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573233 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573208.951394:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573208.951398:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573208.951401:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616334720:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573208.951477:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616334848:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573208.951504:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573208.951518:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573208/real 1788573208] req@ffff9ada4566df80 x1875449616334848/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573233 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573208.951538:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566df80 x1875449616334848/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573233 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573208.951548:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573208.951552:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573208.951555:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616334848:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573208.951563:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616334976:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573208.951581:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573208.951590:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573208/real 1788573208] req@ffff9ada4566c000 x1875449616334976/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573233 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573208.951598:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c000 x1875449616334976/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573233 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573208.951604:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573208.951606:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573208.951608:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616334976:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573209.142907:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573209.463085:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573210.166971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573210.487054:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573211.190954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573211.510989:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573212.215009:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573212.535010:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573213.238986:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573213.559007:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573214.070958:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573214.070970:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573214.070975:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573214.070985:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5126 00000100:00080000:3.0:1788573214.070994:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573214.071000:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573214.071043:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1180 x1875449616335104/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573214.071073:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573214.071077:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573214.071079:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573214.071083:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5126 00000100:00080000:3.0:1788573214.071086:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573214.071090:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573214.071098:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573214.071110:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1880 x1875449616335232/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573214.071118:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616335104:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788573214.071120:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573214.071128:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573214.071130:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573214.071133:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5126 00000100:00080000:3.0:1788573214.071136:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573214.071138:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573214.071141:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573214.071151:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3800 x1875449616335360/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573214.071192:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573214.071252:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573214/real 1788573214] req@ffff9ada604e1180 x1875449616335104/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573239 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573214.071273:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1180 x1875449616335104/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573239 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573214.071286:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573214.071290:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573214.071294:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616335104:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573214.071330:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616335232:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573214.071367:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573214.071382:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573214/real 1788573214] req@ffff9ada604e1880 x1875449616335232/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573239 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573214.071396:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1880 x1875449616335232/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573239 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573214.071405:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573214.071409:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573214.071413:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616335232:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573214.071419:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616335360:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573214.071440:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573214.071451:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573214/real 1788573214] req@ffff9ada604e3800 x1875449616335360/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573239 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573214.071463:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3800 x1875449616335360/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573239 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573214.071480:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573214.071484:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573214.071487:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616335360:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573214.262959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573214.583007:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573215.286994:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573215.607067:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573216.310987:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573216.630978:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573217.334954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573217.655048:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573218.358944:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573218.679047:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573219.191017:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573219.191031:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573219.191036:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573219.191047:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5131 00000100:00080000:3.0:1788573219.191059:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573219.191069:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573219.191131:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0380 x1875449616335488/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573219.191160:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573219.191163:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573219.191164:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573219.191167:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5131 00000100:00080000:3.0:1788573219.191170:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573219.191173:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573219.191183:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573219.191194:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0700 x1875449616335616/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573219.191203:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1788573219.191208:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616335488:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788573219.191208:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573219.191212:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573219.191235:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5131 00000100:00080000:3.0:1788573219.191239:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573219.191243:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573219.191248:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573219.191258:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3480 x1875449616335744/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573219.191326:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573219.191365:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573219/real 1788573219] req@ffff9ada604e0380 x1875449616335488/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573244 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573219.191382:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0380 x1875449616335488/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573244 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573219.191391:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573219.191394:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573219.191398:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616335488:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573219.191421:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616335616:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573219.191438:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573219.191447:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573219/real 1788573219] req@ffff9ada604e0700 x1875449616335616/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573244 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573219.191456:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0700 x1875449616335616/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573244 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573219.191462:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573219.191464:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573219.191466:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616335616:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573219.191470:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616335744:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573219.191483:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573219.191490:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573219/real 1788573219] req@ffff9ada604e3480 x1875449616335744/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573244 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573219.191497:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3480 x1875449616335744/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573244 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573219.191503:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573219.191504:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573219.191506:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616335744:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573219.382965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573219.703103:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573220.406950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573220.726991:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573221.430937:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573221.751016:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573222.454972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573222.775065:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573223.479048:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573223.798988:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573224.311014:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573224.311026:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573224.311031:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573224.311041:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5136 00000100:00080000:2.0:1788573224.311053:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573224.311062:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573224.311105:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627100 x1875449616335872/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573224.311147:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573224.311151:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573224.311154:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573224.311158:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5136 00000100:00080000:2.0:1788573224.311162:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573224.311167:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573224.311177:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573224.311195:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624700 x1875449616336000/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573224.311208:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1788573224.311213:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616335872:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788573224.311214:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573224.311218:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573224.311227:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5136 00000100:00080000:2.0:1788573224.311230:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573224.311234:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573224.311239:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573224.311252:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616336128/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573224.311295:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573224.311342:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573224/real 1788573224] req@ffff9ada60627100 x1875449616335872/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573249 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573224.311363:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627100 x1875449616335872/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573249 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573224.311375:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573224.311379:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573224.311383:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616335872:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573224.311408:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616336000:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573224.311435:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573224.311449:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573224/real 1788573224] req@ffff9ada60624700 x1875449616336000/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573249 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573224.311463:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624700 x1875449616336000/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573249 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573224.311472:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573224.311476:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573224.311480:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616336000:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573224.311487:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616336128:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573224.311507:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573224.311519:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573224/real 1788573224] req@ffff9ada60626680 x1875449616336128/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573249 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573224.311532:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616336128/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573249 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573224.311546:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573224.311550:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573224.311554:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616336128:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573224.502907:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573224.823031:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573225.526966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573225.847035:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573226.550966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573226.871054:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573227.574972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573227.894989:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573228.598920:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573228.919027:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573229.430967:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573229.430978:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573229.430982:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573229.430993:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5141 00000100:00080000:2.0:1788573229.431004:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573229.431012:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573229.431062:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616336256/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573229.431094:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573229.431098:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573229.431100:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573229.431104:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5141 00000100:00080000:2.0:1788573229.431109:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573229.431114:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573229.431125:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573229.431137:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616336384/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573229.431144:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616336256:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788573229.431155:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573229.431158:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573229.431159:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573229.431163:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5141 00000100:00080000:2.0:1788573229.431166:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573229.431169:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573229.431172:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573229.431181:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616336512/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573229.431211:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573229.431256:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573229/real 1788573229] req@ffff9ada60625500 x1875449616336256/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573254 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573229.431272:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616336256/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573254 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573229.431280:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573229.431283:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573229.431286:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616336256:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573229.431305:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616336384:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573229.431322:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573229.431333:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573229/real 1788573229] req@ffff9ada60624380 x1875449616336384/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573254 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573229.431346:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616336384/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573254 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573229.431355:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573229.431359:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573229.431363:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616336384:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573229.431369:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616336512:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573229.431389:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573229.431398:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573229/real 1788573229] req@ffff9ada60626a00 x1875449616336512/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573254 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573229.431406:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616336512/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573254 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573229.431411:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573229.431413:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573229.431415:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616336512:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573229.622967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573229.943173:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573230.646911:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573230.967043:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573231.670971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573231.991022:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573232.694935:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573233.014991:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573233.718977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573234.039035:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573234.550952:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573234.550961:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573234.550965:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573234.550972:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5147 00000100:00080000:3.0:1788573234.550981:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573234.550987:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573234.551020:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3480 x1875449616336640/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573234.551044:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573234.551047:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573234.551048:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573234.551051:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5147 00000100:00080000:3.0:1788573234.551053:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573234.551055:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573234.551062:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573234.551070:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0700 x1875449616336768/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573234.551077:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573234.551080:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573234.551082:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573234.551084:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5147 00000100:00080000:3.0:1788573234.551086:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573234.551089:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573234.551092:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573234.551097:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616336640:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573234.551103:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0380 x1875449616336896/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573234.551153:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573234.551184:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573234/real 1788573234] req@ffff9ada604e3480 x1875449616336640/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573259 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573234.551198:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3480 x1875449616336640/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573259 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573234.551206:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573234.551209:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573234.551212:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616336640:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573234.551218:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616336768:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573234.551232:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573234.551248:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573234/real 1788573234] req@ffff9ada604e0700 x1875449616336768/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573259 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573234.551261:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0700 x1875449616336768/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573259 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573234.551267:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573234.551269:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573234.551271:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616336768:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573234.551292:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616336896:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573234.551313:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573234.551326:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573234/real 1788573234] req@ffff9ada604e0380 x1875449616336896/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573259 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573234.551339:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0380 x1875449616336896/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573259 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573234.551348:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573234.551351:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573234.551354:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616336896:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573234.742945:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573235.063047:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573235.766972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573236.086989:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573236.790994:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573237.111121:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573237.814999:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573238.135028:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573238.838949:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573239.159228:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573239.671001:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573239.671024:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573239.671029:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573239.671040:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5152 00000100:00080000:0.0:1788573239.671051:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573239.671059:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573239.671107:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616337024/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573239.671135:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573239.671138:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573239.671139:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573239.671143:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5152 00000100:00080000:0.0:1788573239.671146:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573239.671151:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573239.671163:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573239.671178:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616337152/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573239.671193:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573239.671202:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:1.0:1788573239.671204:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616337024:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788573239.671205:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573239.671216:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5152 00000100:00080000:0.0:1788573239.671221:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573239.671224:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573239.671230:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573239.671244:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616337280/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573239.671277:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573239.671314:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573239/real 1788573239] req@ffff9ada4557d880 x1875449616337024/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573264 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573239.671328:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616337024/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573264 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573239.671336:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573239.671339:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573239.671342:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616337024:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573239.671363:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616337152:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573239.671379:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573239.671389:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573239/real 1788573239] req@ffff9ada4557f800 x1875449616337152/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573264 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573239.671401:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616337152/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573264 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573239.671410:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573239.671414:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573239.671418:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616337152:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573239.671425:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616337280:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573239.671445:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573239.671454:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573239/real 1788573239] req@ffff9ada4557ca80 x1875449616337280/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573264 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573239.671462:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616337280/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573264 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573239.671468:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573239.671470:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573239.671472:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616337280:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573239.862973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573240.183071:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573240.886921:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573241.207000:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573241.910959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573242.231021:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573242.934932:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573243.254992:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573243.958951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573244.279018:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573244.790943:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573244.790953:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573244.790958:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573244.790968:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5157 00000100:00080000:2.0:1788573244.790980:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573244.790986:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573244.791020:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616337408/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573244.791045:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573244.791049:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573244.791050:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573244.791053:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5157 00000100:00080000:2.0:1788573244.791056:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573244.791058:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573244.791066:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573244.791075:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616337536/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573244.791083:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573244.791086:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573244.791088:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573244.791090:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5157 00000100:00080000:2.0:1788573244.791093:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1788573244.791098:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616337408:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788573244.791098:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573244.791103:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573244.791112:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616337664/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573244.791168:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573244.791202:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573244/real 1788573244] req@ffff9ada60624a80 x1875449616337408/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573269 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573244.791222:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616337408/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573269 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573244.791233:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573244.791238:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573244.791253:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616337408:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573244.791263:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616337536:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573244.791282:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573244.791292:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573244/real 1788573244] req@ffff9ada60625880 x1875449616337536/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573269 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573244.791306:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616337536/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573269 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573244.791311:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573244.791314:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573244.791316:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616337536:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573244.791335:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616337664:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573244.791347:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573244.791353:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573244/real 1788573244] req@ffff9ada60625c00 x1875449616337664/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573269 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573244.791366:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616337664/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573269 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573244.791374:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573244.791377:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573244.791381:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616337664:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573244.982981:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573245.302994:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573246.006971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573246.327063:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573247.030943:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573247.350998:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573248.054925:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573248.375017:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573249.078936:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573249.399012:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573249.910969:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573249.910977:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573249.910981:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573249.910990:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5162 00000100:00080000:0.0:1788573249.910998:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573249.911005:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573249.911038:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c000 x1875449616337792/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573249.911060:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573249.911063:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573249.911064:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573249.911067:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5162 00000100:00080000:0.0:1788573249.911069:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573249.911071:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573249.911079:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573249.911088:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616337920/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573249.911096:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573249.911100:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:1.0:1788573249.911100:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616337792:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788573249.911104:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573249.911111:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5162 00000100:00080000:0.0:1788573249.911113:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573249.911115:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573249.911118:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573249.911128:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616338048/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573249.911198:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616337920:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573249.911226:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616338048:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573250.102951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573250.423040:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573251.126907:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573251.446994:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573252.150932:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573252.471043:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1788573252.983121:0:2373:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:2.0:1788573252.983155:0:2373:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1788573252.983168:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788573252.983218:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788573252.983229:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:1.0:1788573252.983272:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573249/real 1788573252] req@ffff9ada4557f480 x1875449616337920/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573274 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573252.983300:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616337920/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573274 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573252.983313:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573252.983319:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573252.983323:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616337920:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573252.983332:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573249/real 1788573252] req@ffff9ada4557c700 x1875449616338048/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573274 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573252.983346:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616338048/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573274 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573252.983355:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573252.983359:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573252.983362:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616338048:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573252.983369:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573249/real 1788573252] req@ffff9ada4557c000 x1875449616337792/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573274 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573252.983381:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c000 x1875449616337792/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573274 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573252.983388:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573252.983392:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573252.983396:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616337792:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573253.174967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573253.494998:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573254.198940:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573254.519036:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573255.030956:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573255.030967:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573255.030972:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573255.030982:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5167 00000100:00080000:0.0:1788573255.030992:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573255.031015:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573255.031054:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ce00 x1875449616338176/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573255.031085:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573255.031089:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573255.031092:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573255.031096:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5167 00000100:00080000:0.0:1788573255.031101:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573255.031106:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573255.031116:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573255.031139:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c380 x1875449616338304/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573255.031149:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573255.031172:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573255.031174:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573255.031178:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5167 00000100:00080000:0.0:1788573255.031181:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573255.031185:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573255.031191:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573255.031191:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616338176:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788573255.031208:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c000 x1875449616338432/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573255.031263:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573255.031297:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573255/real 1788573255] req@ffff9ada4557ce00 x1875449616338176/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573280 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573255.031316:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ce00 x1875449616338176/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573280 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573255.031326:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573255.031330:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573255.031335:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616338176:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573255.031342:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616338304:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573255.031366:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573255.031379:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573255/real 1788573255] req@ffff9ada4557c380 x1875449616338304/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573280 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573255.031391:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c380 x1875449616338304/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573280 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573255.031399:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573255.031402:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573255.031405:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616338304:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573255.031430:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616338432:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573255.031455:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573255.031466:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573255/real 1788573255] req@ffff9ada4557c000 x1875449616338432/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573280 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573255.031479:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c000 x1875449616338432/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573280 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573255.031487:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573255.031490:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573255.031494:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616338432:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573255.222961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573255.542991:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573256.246961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573256.567023:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573257.270989:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573257.591059:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573258.294920:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573258.615117:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573259.318958:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573259.639011:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788573260.151008:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573260.151016:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788573260.151020:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573260.151028:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5172 00000100:00080000:1.0:1788573260.151038:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573260.151045:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788573260.151080:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566dc00 x1875449616338560/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573260.151098:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573260.151102:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788573260.151104:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573260.151107:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5172 00000100:00080000:1.0:1788573260.151110:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573260.151113:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573260.151120:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616338560:192.168.202.113@tcp:250:kworker.0 00000008:00100000:1.0:1788573260.151121:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573260.151135:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f800 x1875449616338688/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573260.151143:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573260.151146:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788573260.151147:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573260.151150:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5172 00000100:00080000:1.0:1788573260.151152:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573260.151154:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573260.151157:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573260.151166:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ed80 x1875449616338816/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573260.151198:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573260.151228:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573260/real 1788573260] req@ffff9ada4566dc00 x1875449616338560/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573285 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573260.151246:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566dc00 x1875449616338560/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573285 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573260.151253:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573260.151256:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573260.151259:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616338560:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573260.151277:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616338688:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573260.151292:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573260.151299:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573260/real 1788573260] req@ffff9ada4566f800 x1875449616338688/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573285 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573260.151307:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f800 x1875449616338688/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573285 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573260.151313:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573260.151315:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573260.151317:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616338688:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573260.151320:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616338816:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573260.151331:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573260.151337:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573260/real 1788573260] req@ffff9ada4566ed80 x1875449616338816/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573285 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573260.151344:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ed80 x1875449616338816/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573285 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573260.151354:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573260.151356:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573260.151358:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616338816:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573260.342956:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573260.663023:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573261.366958:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573261.687051:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573262.390932:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573262.711004:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573263.414957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573263.735062:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573264.438970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573264.758970:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788573265.270979:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573265.270986:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788573265.270990:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573265.270999:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5177 00000100:00080000:1.0:1788573265.271008:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573265.271014:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788573265.271048:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ea00 x1875449616338944/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573265.271079:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573265.271081:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788573265.271083:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573265.271086:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5177 00000100:00080000:1.0:1788573265.271088:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573265.271090:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573265.271097:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573265.271106:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e300 x1875449616339072/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573265.271113:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573265.271115:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788573265.271117:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573265.271119:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5177 00000100:00080000:1.0:1788573265.271121:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573265.271123:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573265.271127:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573265.271132:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616338944:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573265.271137:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f480 x1875449616339200/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573265.271190:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573265.271227:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573265/real 1788573265] req@ffff9ada4566ea00 x1875449616338944/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573290 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573265.271242:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ea00 x1875449616338944/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573290 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573265.271249:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573265.271256:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573265.271259:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616338944:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573265.271266:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616339072:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573265.271289:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573265.271303:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573265/real 1788573265] req@ffff9ada4566e300 x1875449616339072/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573290 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573265.271318:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e300 x1875449616339072/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573290 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573265.271325:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573265.271328:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573265.271330:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616339072:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573265.271348:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616339200:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573265.271378:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573265.271387:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573265/real 1788573265] req@ffff9ada4566f480 x1875449616339200/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573290 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573265.271395:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f480 x1875449616339200/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573290 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573265.271401:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573265.271403:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573265.271405:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616339200:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573265.462939:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573265.783043:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573266.486956:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573266.807071:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573267.510990:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573267.831020:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573268.535010:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573268.855103:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573269.558937:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573269.879081:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573270.390950:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573270.390958:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573270.390962:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573270.390970:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5182 00000100:00080000:0.0:1788573270.390979:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573270.390986:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573270.391017:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616339328/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573270.391039:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573270.391043:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573270.391045:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573270.391049:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5182 00000100:00080000:0.0:1788573270.391053:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573270.391057:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573270.391067:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573270.391078:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616339328:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788573270.391079:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616339456/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573270.391089:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573270.391092:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573270.391094:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573270.391097:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5182 00000100:00080000:0.0:1788573270.391101:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573270.391104:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573270.391108:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573270.391116:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616339584/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573270.391136:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573270.391169:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573270/real 1788573270] req@ffff9ada4557c700 x1875449616339328/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573295 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573270.391184:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616339328/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573295 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573270.391193:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573270.391195:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573270.391198:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616339328:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573270.391217:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616339456:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573270.391234:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573270.391242:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573270/real 1788573270] req@ffff9ada4557f480 x1875449616339456/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573295 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573270.391250:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616339456/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573295 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573270.391255:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573270.391257:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573270.391259:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616339456:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573270.391263:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616339584:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573270.391273:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573270.391279:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573270/real 1788573270] req@ffff9ada4557ca80 x1875449616339584/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573295 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573270.391286:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616339584/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573295 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573270.391291:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573270.391293:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573270.391294:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616339584:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573270.582952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573270.903027:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573271.606968:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573271.927044:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573272.630965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573272.951088:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573273.654968:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573273.975069:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573274.678959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573274.998988:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788573275.510941:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573275.510952:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788573275.510957:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573275.510966:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5188 00000100:00080000:1.0:1788573275.510976:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573275.510984:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788573275.511016:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ca80 x1875449616339712/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573275.511044:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573275.511052:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788573275.511054:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573275.511058:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5188 00000100:00080000:1.0:1788573275.511062:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573275.511066:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573275.511074:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573275.511086:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d180 x1875449616339840/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788573275.511089:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616339712:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788573275.511105:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573275.511109:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788573275.511111:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573275.511116:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5188 00000100:00080000:1.0:1788573275.511120:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573275.511124:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573275.511129:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573275.511139:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d500 x1875449616339968/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573275.511150:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573275.511183:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573275/real 1788573275] req@ffff9ada4566ca80 x1875449616339712/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573300 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573275.511197:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ca80 x1875449616339712/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573300 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573275.511204:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573275.511206:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573275.511209:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616339712:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573275.511226:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616339840:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573275.511257:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573275.511267:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573275/real 1788573275] req@ffff9ada4566d180 x1875449616339840/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573300 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573275.511275:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d180 x1875449616339840/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573300 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573275.511280:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573275.511282:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573275.511284:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616339840:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573275.511288:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616339968:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573275.511300:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573275.511305:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573275/real 1788573275] req@ffff9ada4566d500 x1875449616339968/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573300 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573275.511312:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d500 x1875449616339968/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573300 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573275.511317:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573275.511318:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573275.511320:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616339968:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573275.702971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573276.022990:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573276.727881:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573277.046990:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573277.750934:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573278.071033:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573278.774955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573279.095030:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573279.798931:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573280.119035:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573280.630983:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573280.630993:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573280.630998:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573280.631008:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5193 00000100:00080000:0.0:1788573280.631018:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573280.631025:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573280.631060:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616340096/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573280.631101:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573280.631104:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573280.631106:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573280.631111:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5193 00000100:00080000:0.0:1788573280.631114:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573280.631118:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573280.631127:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573280.631140:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616340224/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573280.631150:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1788573280.631153:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616340096:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788573280.631154:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573280.631158:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573280.631164:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5193 00000100:00080000:0.0:1788573280.631167:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573280.631171:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573280.631175:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573280.631202:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616340352/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573280.631234:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573280.631267:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573280/real 1788573280] req@ffff9ada4557f800 x1875449616340096/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573305 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573280.631285:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616340096/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573305 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573280.631295:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573280.631299:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573280.631302:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616340096:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573280.631323:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616340224:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573280.631346:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573280.631358:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573280/real 1788573280] req@ffff9ada4557d880 x1875449616340224/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573305 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573280.631368:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616340224/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573305 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573280.631374:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573280.631375:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573280.631377:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616340224:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573280.631381:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616340352:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573280.631394:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573280.631400:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573280/real 1788573280] req@ffff9ada4557fb80 x1875449616340352/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573305 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573280.631406:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616340352/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573305 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573280.631411:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573280.631413:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573280.631416:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616340352:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573280.822944:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573281.142966:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573281.846905:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573282.166982:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573282.870962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573283.190969:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573283.894959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573284.214998:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573284.918920:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573285.239020:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573285.750916:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573285.750924:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573285.750927:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573285.750935:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5198 00000100:00080000:0.0:1788573285.750942:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573285.750948:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573285.750977:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616340480/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573285.750998:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573285.751001:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573285.751003:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573285.751006:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5198 00000100:00080000:0.0:1788573285.751009:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573285.751012:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573285.751021:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573285.751032:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616340480:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788573285.751034:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616340608/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573285.751043:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573285.751045:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573285.751047:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573285.751049:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5198 00000100:00080000:0.0:1788573285.751052:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573285.751055:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573285.751059:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573285.751067:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616340736/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573285.751083:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573285.751112:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573285/real 1788573285] req@ffff9ada4557dc00 x1875449616340480/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573310 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573285.751147:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616340480/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573310 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573285.751155:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573285.751157:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573285.751159:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616340480:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573285.751176:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616340608:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573285.751192:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573285.751200:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573285/real 1788573285] req@ffff9ada4557e300 x1875449616340608/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573310 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573285.751207:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616340608/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573310 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573285.751212:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573285.751213:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573285.751215:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616340608:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573285.751218:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616340736:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573285.751229:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573285.751234:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573285/real 1788573285] req@ffff9ada4557ed80 x1875449616340736/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573310 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573285.751241:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616340736/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573310 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573285.751245:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573285.751246:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573285.751248:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616340736:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573285.942961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573286.262985:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573286.966917:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573287.287013:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573287.990947:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573288.311007:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573289.014909:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573289.335009:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573290.038923:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573290.359092:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573290.870988:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573290.870999:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573290.871003:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573290.871012:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5203 00000100:00080000:3.0:1788573290.871021:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573290.871028:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573290.871077:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3800 x1875449616340864/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573290.871105:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573290.871108:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573290.871110:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573290.871112:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5203 00000100:00080000:3.0:1788573290.871115:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573290.871118:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573290.871126:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573290.871136:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1880 x1875449616340992/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573290.871145:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573290.871148:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573290.871150:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573290.871155:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5203 00000100:00080000:3.0:1788573290.871158:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573290.871164:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788573290.871166:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616340864:192.168.202.113@tcp:250:kworker.0 00000008:00100000:3.0:1788573290.871170:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573290.871185:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1180 x1875449616341120/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573290.871237:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573290.871286:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573290/real 1788573290] req@ffff9ada604e3800 x1875449616340864/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573315 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573290.871307:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3800 x1875449616340864/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573315 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573290.871319:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573290.871324:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573290.871328:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616340864:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573290.871337:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616340992:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573290.871363:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573290.871378:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573290/real 1788573290] req@ffff9ada604e1880 x1875449616340992/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573315 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573290.871391:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1880 x1875449616340992/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573315 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573290.871407:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573290.871410:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573290.871414:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616340992:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573290.871439:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616341120:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573290.871463:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573290.871476:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573290/real 1788573290] req@ffff9ada604e1180 x1875449616341120/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573315 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573290.871490:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1180 x1875449616341120/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573315 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573290.871499:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573290.871502:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573290.871506:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616341120:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573291.062914:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573291.383043:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573292.086946:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573292.406966:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573293.110946:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573293.431107:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573294.134964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573294.455076:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573295.158952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573295.479081:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573295.990990:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573295.991014:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573295.991018:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573295.991028:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5208 00000100:00080000:0.0:1788573295.991040:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573295.991048:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573295.991089:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616341248/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573295.991131:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573295.991136:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573295.991138:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573295.991141:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5208 00000100:00080000:0.0:1788573295.991144:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573295.991148:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573295.991157:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573295.991169:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d180 x1875449616341376/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573295.991178:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1788573295.991181:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616341248:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788573295.991182:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573295.991186:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573295.991195:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5208 00000100:00080000:0.0:1788573295.991197:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573295.991199:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573295.991203:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573295.991212:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557df80 x1875449616341504/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573295.991256:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573295.991294:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573295/real 1788573295] req@ffff9ada4557e680 x1875449616341248/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573320 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573295.991313:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616341248/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573320 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573295.991324:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573295.991328:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573295.991334:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616341248:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573295.991357:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616341376:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573295.991376:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573295.991385:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573295/real 1788573295] req@ffff9ada4557d180 x1875449616341376/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573320 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573295.991393:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d180 x1875449616341376/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573320 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573295.991399:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573295.991401:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573295.991403:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616341376:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573295.991407:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616341504:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573295.991418:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573295.991424:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573295/real 1788573295] req@ffff9ada4557df80 x1875449616341504/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573320 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573295.991432:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557df80 x1875449616341504/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573320 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573295.991437:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573295.991439:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573295.991441:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616341504:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573296.182976:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573296.503040:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573297.206957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573297.527050:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573298.231081:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573298.551056:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573299.254919:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573299.575072:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573300.279011:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573300.599079:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573301.110993:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573301.111007:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573301.111012:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573301.111022:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5213 00000100:00080000:2.0:1788573301.111033:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573301.111041:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573301.111079:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616341632/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573301.111117:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573301.111122:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573301.111125:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573301.111131:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5213 00000100:00080000:2.0:1788573301.111134:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573301.111139:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573301.111149:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573301.111164:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616341760/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788573301.111177:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616341632:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788573301.111181:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573301.111193:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573301.111194:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573301.111199:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5213 00000100:00080000:2.0:1788573301.111203:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573301.111208:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573301.111216:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573301.111230:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616341888/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573301.111282:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573301.111330:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573301/real 1788573301] req@ffff9ada60625f80 x1875449616341632/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573326 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573301.111354:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616341632/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573326 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573301.111368:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573301.111374:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573301.111378:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616341632:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573301.111409:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616341760:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573301.111431:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573301.111441:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573301/real 1788573301] req@ffff9ada60624e00 x1875449616341760/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573326 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573301.111450:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616341760/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573326 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573301.111456:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573301.111459:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573301.111461:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616341760:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573301.111466:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616341888:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573301.111484:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573301.111491:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573301/real 1788573301] req@ffff9ada60627480 x1875449616341888/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573326 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573301.111499:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616341888/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573326 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573301.111504:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573301.111506:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573301.111508:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616341888:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573301.303017:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573301.623267:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573302.327027:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573302.647107:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573303.350937:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573303.670994:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573304.374976:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573304.694992:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573305.398986:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573305.719007:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573306.230952:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573306.230962:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573306.230967:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573306.230978:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5218 00000100:00080000:2.0:1788573306.230989:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573306.230996:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573306.231039:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616342016/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573306.231070:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573306.231074:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573306.231076:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573306.231079:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5218 00000100:00080000:2.0:1788573306.231083:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573306.231087:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573306.231097:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573306.231112:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616342144/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788573306.231121:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616342016:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788573306.231129:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573306.231133:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573306.231135:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573306.231139:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5218 00000100:00080000:2.0:1788573306.231142:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573306.231145:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573306.231151:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573306.231161:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616342272/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573306.231179:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573306.231212:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573306/real 1788573306] req@ffff9ada60625180 x1875449616342016/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573331 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573306.231226:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616342016/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573331 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573306.231235:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573306.231238:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573306.231240:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616342016:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573306.231260:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616342144:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573306.231274:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573306.231283:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573306/real 1788573306] req@ffff9ada60626d80 x1875449616342144/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573331 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573306.231292:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616342144/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573331 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573306.231298:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573306.231300:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573306.231302:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616342144:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573306.231306:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616342272:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573306.231316:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573306.231321:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573306/real 1788573306] req@ffff9ada60627b80 x1875449616342272/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573331 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573306.231329:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616342272/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573331 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573306.231338:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573306.231341:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573306.231345:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616342272:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573306.422940:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573306.743028:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573307.446986:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573307.767057:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573308.470941:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573308.791126:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573309.494974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573309.815047:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573310.518970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573310.839016:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573311.350971:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573311.350982:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573311.350987:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573311.350998:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5223 00000100:00080000:2.0:1788573311.351009:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573311.351018:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573311.351053:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627800 x1875449616342400/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573311.351083:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573311.351087:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573311.351089:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573311.351093:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5223 00000100:00080000:2.0:1788573311.351098:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573311.351103:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573311.351113:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573311.351129:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626300 x1875449616342528/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788573311.351129:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616342400:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788573311.351147:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573311.351151:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573311.351153:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573311.351157:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5223 00000100:00080000:2.0:1788573311.351163:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573311.351167:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573311.351172:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573311.351185:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616342656/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573311.351190:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573311.351221:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573311/real 1788573311] req@ffff9ada60627800 x1875449616342400/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573336 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573311.351249:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627800 x1875449616342400/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573336 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573311.351259:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573311.351261:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573311.351264:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616342400:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573311.351281:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616342528:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573311.351297:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573311.351305:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573311/real 1788573311] req@ffff9ada60626300 x1875449616342528/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573336 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573311.351313:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626300 x1875449616342528/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573336 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573311.351319:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573311.351321:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573311.351323:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616342528:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573311.351326:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616342656:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573311.351336:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573311.351341:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573311/real 1788573311] req@ffff9ada60627b80 x1875449616342656/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573336 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573311.351349:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616342656/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573336 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573311.351358:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573311.351360:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573311.351362:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616342656:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573311.542978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573311.863018:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573312.566953:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573312.887044:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573313.590961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573313.910990:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573314.615001:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573314.935032:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573315.639001:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573315.959068:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573316.471014:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573316.471025:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573316.471031:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573316.471043:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5229 00000100:00080000:0.0:1788573316.471055:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573316.471064:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573316.471109:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d500 x1875449616342784/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573316.471132:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573316.471135:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573316.471137:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573316.471140:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5229 00000100:00080000:0.0:1788573316.471142:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573316.471145:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573316.471156:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573316.471165:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616342912/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573316.471173:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573316.471176:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573316.471177:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573316.471180:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5229 00000100:00080000:0.0:1788573316.471182:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573316.471184:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573316.471187:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573316.471198:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616342784:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788573316.471199:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557df80 x1875449616343040/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788573316.471300:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616342912:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573316.471334:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616343040:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573316.662973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573316.983053:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573317.686958:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573318.006974:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573318.710967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573319.031046:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1788573319.543214:0:2371:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:0.0:1788573319.543267:0:2371:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1788573319.543291:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:0.0:1788573319.543353:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:0.0:1788573319.543364:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:3.0:1788573319.543484:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573316/real 1788573319] req@ffff9ada4557df80 x1875449616343040/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573341 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573319.543518:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557df80 x1875449616343040/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573341 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573319.543533:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573319.543539:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573319.543544:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616343040:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573319.543553:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573316/real 1788573319] req@ffff9ada4557d500 x1875449616342784/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573341 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573319.543574:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d500 x1875449616342784/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573341 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573319.543582:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573319.543587:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573319.543591:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616342784:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573319.543611:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573316/real 1788573319] req@ffff9ada4557ea00 x1875449616342912/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573341 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573319.543624:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616342912/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573341 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573319.543632:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573319.543637:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573319.543640:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616342912:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573319.734969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573320.055023:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573320.758972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573321.078999:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573321.591091:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573321.591101:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573321.591105:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573321.591118:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5234 00000100:00080000:0.0:1788573321.591132:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573321.591141:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573321.591190:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d180 x1875449616343168/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573321.591236:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573321.591241:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573321.591243:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573321.591248:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5234 00000100:00080000:0.0:1788573321.591252:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573321.591264:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573321.591281:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573321.591301:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616343296/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573321.591316:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573321.591321:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573321.591324:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573321.591330:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5234 00000100:00080000:0.0:1788573321.591335:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573321.591344:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573321.591350:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616343168:192.168.202.113@tcp:250:kworker.0 00000008:00100000:0.0:1788573321.591352:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573321.591382:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616343424/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573321.591439:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573321.591481:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573321/real 1788573321] req@ffff9ada4557d180 x1875449616343168/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573346 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573321.591502:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d180 x1875449616343168/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573346 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573321.591515:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573321.591521:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573321.591526:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616343168:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573321.591553:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616343296:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573321.591573:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573321.591584:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573321/real 1788573321] req@ffff9ada4557e680 x1875449616343296/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573346 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573321.591597:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616343296/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573346 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573321.591606:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573321.591609:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573321.591613:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616343296:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573321.591619:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616343424:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573321.591635:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573321.591643:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573321/real 1788573321] req@ffff9ada4557ed80 x1875449616343424/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573346 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573321.591655:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616343424/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573346 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573321.591663:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573321.591667:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573321.591670:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616343424:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573321.782955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573322.103048:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573322.806992:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573323.126973:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573323.830922:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573324.151031:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573324.854975:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573325.174985:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573325.878935:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573326.199053:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573326.711015:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573326.711028:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573326.711033:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573326.711044:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5239 00000100:00080000:2.0:1788573326.711056:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573326.711064:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573326.711107:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616343552/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573326.711135:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573326.711139:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573326.711140:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573326.711143:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5239 00000100:00080000:2.0:1788573326.711146:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573326.711149:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573326.711158:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573326.711168:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616343680/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573326.711176:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573326.711179:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573326.711181:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573326.711183:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5239 00000100:00080000:2.0:1788573326.711185:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573326.711188:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573326.711191:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573326.711201:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616343808/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788573326.711202:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616343552:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788573326.711279:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573326.711316:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573326/real 1788573326] req@ffff9ada60626d80 x1875449616343552/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573351 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573326.711333:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616343552/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573351 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573326.711342:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573326.711346:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573326.711348:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616343552:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573326.711354:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616343680:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573326.711379:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573326.711388:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573326/real 1788573326] req@ffff9ada60625180 x1875449616343680/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573351 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573326.711396:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616343680/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573351 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573326.711404:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573326.711407:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573326.711411:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616343680:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573326.711442:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616343808:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573326.711463:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573326.711480:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573326/real 1788573326] req@ffff9ada60627480 x1875449616343808/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573351 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573326.711489:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616343808/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573351 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573326.711495:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573326.711497:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573326.711499:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616343808:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573326.903003:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573327.223046:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573327.926983:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573328.247071:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573328.950952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573329.271064:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573329.975013:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573330.295106:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573330.999012:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573331.319033:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573331.830974:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573331.830994:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573331.831001:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573331.831015:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5244 00000100:00080000:0.0:1788573331.831029:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573331.831037:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573331.831088:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616343936/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573331.831124:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573331.831129:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573331.831131:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573331.831136:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5244 00000100:00080000:0.0:1788573331.831140:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573331.831145:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573331.831158:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573331.831177:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616344064/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788573331.831190:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616343936:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788573331.831192:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573331.831200:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573331.831203:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573331.831207:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5244 00000100:00080000:0.0:1788573331.831212:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573331.831217:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573331.831223:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573331.831237:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616344192/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573331.831275:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573331.831313:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573331/real 1788573331] req@ffff9ada4557e300 x1875449616343936/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573356 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573331.831339:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616343936/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573356 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573331.831349:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573331.831352:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573331.831356:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616343936:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573331.831379:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616344064:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573331.831398:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573331.831407:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573331/real 1788573331] req@ffff9ada4557dc00 x1875449616344064/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573356 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573331.831416:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616344064/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573356 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573331.831422:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573331.831424:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573331.831427:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616344064:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573331.831430:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616344192:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573331.831446:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573331.831457:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573331/real 1788573331] req@ffff9ada4557fb80 x1875449616344192/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573356 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573331.831470:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616344192/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573356 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573331.831489:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573331.831497:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573331.831500:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616344192:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573332.022980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573332.343036:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573333.046988:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573333.366996:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573334.070952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573334.390997:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573335.094985:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573335.415019:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573336.118954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573336.439067:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573336.950981:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573336.950990:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573336.950993:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573336.951002:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5249 00000100:00080000:2.0:1788573336.951011:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573336.951017:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573336.951051:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616344320/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573336.951074:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573336.951077:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573336.951078:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573336.951081:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5249 00000100:00080000:2.0:1788573336.951083:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573336.951085:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573336.951098:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573336.951108:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616344448/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573336.951117:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573336.951119:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573336.951121:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1788573336.951126:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616344320:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788573336.951127:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5249 00000100:00080000:2.0:1788573336.951130:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573336.951132:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573336.951138:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573336.951146:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616344576/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573336.951190:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573336.951226:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573336/real 1788573336] req@ffff9ada60624e00 x1875449616344320/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573361 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573336.951245:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616344320/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573361 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573336.951256:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573336.951259:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573336.951264:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616344320:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573336.951286:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616344448:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573336.951309:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573336.951320:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573336/real 1788573336] req@ffff9ada60625f80 x1875449616344448/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573361 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573336.951330:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616344448/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573361 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573336.951336:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573336.951338:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573336.951340:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616344448:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573336.951344:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616344576:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573336.951356:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573336.951363:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573336/real 1788573336] req@ffff9ada60625c00 x1875449616344576/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573361 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573336.951370:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616344576/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573361 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573336.951380:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573336.951382:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573336.951384:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616344576:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573337.142969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573337.463046:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573338.166942:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573338.486996:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573339.190944:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573339.511073:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573340.214948:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573340.535022:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573341.238950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573341.559031:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573342.071015:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573342.071028:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573342.071032:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573342.071043:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5254 00000100:00080000:0.0:1788573342.071055:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573342.071062:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573342.071112:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616344704/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573342.071136:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573342.071140:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573342.071141:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573342.071144:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5254 00000100:00080000:0.0:1788573342.071147:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573342.071149:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573342.071159:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573342.071167:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616344832/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573342.071174:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573342.071177:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573342.071178:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573342.071182:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5254 00000100:00080000:0.0:1788573342.071184:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573342.071187:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573342.071190:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573342.071196:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616344960/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788573342.071244:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616344704:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788573342.071310:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573342.071343:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573342/real 1788573342] req@ffff9ada4557d880 x1875449616344704/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573367 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573342.071358:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616344704/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573367 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573342.071367:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573342.071371:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573342.071373:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616344704:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573342.071379:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616344832:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573342.071392:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573342.071400:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573342/real 1788573342] req@ffff9ada4557f800 x1875449616344832/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573367 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573342.071408:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616344832/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573367 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573342.071417:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573342.071420:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573342.071422:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616344832:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573342.071426:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616344960:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573342.071435:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573342.071441:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573342/real 1788573342] req@ffff9ada4557ca80 x1875449616344960/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573367 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573342.071448:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616344960/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573367 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573342.071453:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573342.071455:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573342.071457:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616344960:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573342.263005:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573342.583042:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573343.287007:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573343.606968:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573344.310966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573344.630964:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573345.334947:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573345.655001:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573346.358955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573346.678999:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573347.190980:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573347.190988:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573347.190992:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573347.191000:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5259 00000100:00080000:3.0:1788573347.191009:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573347.191015:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573347.191045:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2300 x1875449616345088/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573347.191069:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573347.191072:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573347.191073:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573347.191076:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5259 00000100:00080000:3.0:1788573347.191078:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573347.191080:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573347.191088:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573347.191095:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2d80 x1875449616345216/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573347.191101:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573347.191104:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573347.191105:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573347.191108:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5259 00000100:00080000:3.0:1788573347.191110:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573347.191112:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573347.191116:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573347.191122:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0a80 x1875449616345344/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573347.191177:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616345088:192.168.202.113@tcp:250:kworker.0 00000800:00000100:1.0:1788573347.191283:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573347.191336:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573347/real 1788573347] req@ffff9ada604e2300 x1875449616345088/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573372 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573347.191369:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2300 x1875449616345088/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573372 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573347.191384:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573347.191389:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573347.191394:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616345088:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573347.191403:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616345216:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573347.191429:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573347.191452:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573347/real 1788573347] req@ffff9ada604e2d80 x1875449616345216/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573372 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573347.191467:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2d80 x1875449616345216/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573372 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573347.191477:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573347.191480:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573347.191484:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616345216:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573347.191491:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616345344:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573347.191530:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573347.191544:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573347/real 1788573347] req@ffff9ada604e0a80 x1875449616345344/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573372 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573347.191564:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0a80 x1875449616345344/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573372 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573347.191572:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573347.191576:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573347.191579:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616345344:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573347.382980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573347.703023:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573348.406995:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573348.727067:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573349.430994:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573349.751050:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573350.455001:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573350.775038:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573351.479018:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573351.799082:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573352.310964:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573352.310972:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573352.310976:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573352.310984:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5264 00000100:00080000:3.0:1788573352.310994:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573352.311012:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573352.311045:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0000 x1875449616345472/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573352.311068:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573352.311070:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573352.311072:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573352.311074:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5264 00000100:00080000:3.0:1788573352.311076:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573352.311079:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573352.311086:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573352.311106:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1c00 x1875449616345600/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573352.311118:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573352.311122:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573352.311124:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1788573352.311132:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616345472:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788573352.311132:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5264 00000100:00080000:3.0:1788573352.311139:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573352.311144:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573352.311149:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573352.311160:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3b80 x1875449616345728/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573352.311199:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573352.311232:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573352/real 1788573352] req@ffff9ada604e0000 x1875449616345472/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573377 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573352.311246:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0000 x1875449616345472/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573377 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573352.311254:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573352.311257:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573352.311260:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616345472:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573352.311265:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616345600:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573352.311280:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573352.311288:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573352/real 1788573352] req@ffff9ada604e1c00 x1875449616345600/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573377 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573352.311296:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1c00 x1875449616345600/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573377 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573352.311301:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573352.311303:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573352.311305:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616345600:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573352.311323:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616345728:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573352.311339:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573352.311347:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573352/real 1788573352] req@ffff9ada604e3b80 x1875449616345728/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573377 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573352.311355:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3b80 x1875449616345728/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573377 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573352.311375:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573352.311377:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573352.311379:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616345728:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573352.503052:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573352.823023:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573353.526954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573353.847066:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573354.550956:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573354.871041:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573355.574939:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573355.895061:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573356.598999:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573356.919020:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573357.430928:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573357.430937:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573357.430941:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573357.430949:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5269 00000100:00080000:3.0:1788573357.430959:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573357.430965:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573357.430998:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3100 x1875449616345856/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573357.431020:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573357.431023:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573357.431025:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573357.431027:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5269 00000100:00080000:3.0:1788573357.431030:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573357.431032:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573357.431039:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573357.431048:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1f80 x1875449616345984/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573357.431054:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573357.431057:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573357.431059:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573357.431061:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5269 00000100:00080000:3.0:1788573357.431063:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573357.431065:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573357.431068:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573357.431073:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2680 x1875449616346112/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573357.431104:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616345856:192.168.202.113@tcp:250:kworker.0 00000800:00000100:1.0:1788573357.431159:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573357.431189:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573357/real 1788573357] req@ffff9ada604e3100 x1875449616345856/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573382 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573357.431203:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3100 x1875449616345856/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573382 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573357.431213:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573357.431218:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573357.431222:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616345856:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573357.431230:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616345984:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573357.431252:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573357.431266:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573357/real 1788573357] req@ffff9ada604e1f80 x1875449616345984/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573382 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573357.431279:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1f80 x1875449616345984/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573382 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573357.431286:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573357.431288:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573357.431291:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616345984:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573357.431295:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616346112:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573357.431311:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573357.431318:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573357/real 1788573357] req@ffff9ada604e2680 x1875449616346112/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573382 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573357.431326:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2680 x1875449616346112/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573382 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573357.431331:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573357.431333:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573357.431334:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616346112:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573357.622950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573357.943033:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573358.647000:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573358.967077:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573359.670961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573359.991052:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573360.694955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573361.015074:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573361.718978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573362.039014:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573362.550975:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573362.550986:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573362.550991:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573362.551002:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5275 00000100:00080000:2.0:1788573362.551014:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573362.551022:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573362.551073:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616346240/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573362.551100:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573362.551103:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573362.551105:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573362.551108:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5275 00000100:00080000:2.0:1788573362.551110:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573362.551112:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573362.551123:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573362.551134:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616346368/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573362.551142:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573362.551145:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573362.551146:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573362.551149:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5275 00000100:00080000:2.0:1788573362.551151:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573362.551153:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573362.551158:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573362.551168:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616346496/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573362.551170:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616346240:192.168.202.113@tcp:250:kworker.0 00000800:00000100:1.0:1788573362.551248:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573362.551281:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573362/real 1788573362] req@ffff9ada60625880 x1875449616346240/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573387 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573362.551295:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616346240/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573387 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573362.551305:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573362.551308:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573362.551311:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616346240:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573362.551319:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616346368:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573362.551358:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573362.551373:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573362/real 1788573362] req@ffff9ada60624a80 x1875449616346368/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573387 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573362.551386:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616346368/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573387 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573362.551392:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573362.551395:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573362.551397:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616346368:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573362.551418:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616346496:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573362.551434:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573362.551442:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573362/real 1788573362] req@ffff9ada60626a00 x1875449616346496/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573387 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573362.551449:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616346496/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573387 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573362.551455:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573362.551457:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573362.551459:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616346496:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573362.742965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573363.063108:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573363.767032:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573364.087012:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573364.790962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573365.111047:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573365.814999:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573366.135184:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573366.838964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573367.159058:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573367.670922:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573367.670930:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573367.670934:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573367.670942:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5280 00000100:00080000:2.0:1788573367.670951:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573367.670957:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573367.670993:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616346624/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573367.671017:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573367.671020:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573367.671022:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573367.671024:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5280 00000100:00080000:2.0:1788573367.671027:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573367.671030:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573367.671037:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573367.671046:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616346752/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573367.671052:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573367.671054:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573367.671056:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573367.671058:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5280 00000100:00080000:2.0:1788573367.671059:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573367.671061:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573367.671064:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573367.671069:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616346880/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573367.671104:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616346624:192.168.202.113@tcp:250:kworker.0 00000800:00000100:1.0:1788573367.671158:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573367.671188:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573367/real 1788573367] req@ffff9ada60624380 x1875449616346624/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573392 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573367.671202:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616346624/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573392 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573367.671210:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573367.671213:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573367.671217:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616346624:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573367.671225:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616346752:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573367.671247:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573367.671260:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573367/real 1788573367] req@ffff9ada60625500 x1875449616346752/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573392 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573367.671274:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616346752/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573392 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573367.671283:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573367.671286:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573367.671288:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616346752:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573367.671293:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616346880:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573367.671309:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573367.671317:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573367/real 1788573367] req@ffff9ada60626680 x1875449616346880/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573392 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573367.671325:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616346880/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573392 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573367.671330:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573367.671332:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573367.671334:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616346880:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573367.862960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573368.183078:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573368.886958:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573369.207012:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573369.910960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573370.231043:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573370.934980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573371.255033:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573371.958977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573372.279003:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573372.790947:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573372.790958:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573372.790962:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573372.790970:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5285 00000100:00080000:3.0:1788573372.790979:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573372.790986:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573372.791020:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1500 x1875449616347008/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573372.791041:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573372.791044:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573372.791046:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573372.791048:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5285 00000100:00080000:3.0:1788573372.791050:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573372.791052:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573372.791059:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573372.791069:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0e00 x1875449616347136/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573372.791077:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573372.791079:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573372.791081:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1788573372.791087:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616347008:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788573372.791087:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5285 00000100:00080000:3.0:1788573372.791090:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573372.791095:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573372.791100:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573372.791106:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2a00 x1875449616347264/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573372.791157:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573372.791189:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573372/real 1788573372] req@ffff9ada604e1500 x1875449616347008/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573397 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573372.791204:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1500 x1875449616347008/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573397 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573372.791212:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573372.791214:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573372.791218:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616347008:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573372.791238:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616347136:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573372.791253:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573372.791261:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573372/real 1788573372] req@ffff9ada604e0e00 x1875449616347136/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573397 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573372.791284:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0e00 x1875449616347136/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573397 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573372.791293:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573372.791297:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573372.791301:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616347136:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573372.791308:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616347264:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573372.791328:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573372.791336:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573372/real 1788573372] req@ffff9ada604e2a00 x1875449616347264/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573397 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573372.791344:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2a00 x1875449616347264/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573397 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573372.791349:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573372.791351:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573372.791353:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616347264:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573372.982945:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573373.303104:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573374.006935:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573374.327009:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573375.030927:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573375.351000:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573376.054961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573376.375077:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573377.078986:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573377.399013:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573377.911032:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573377.911040:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573377.911044:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573377.911052:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5290 00000100:00080000:2.0:1788573377.911061:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573377.911068:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573377.911102:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624700 x1875449616347392/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573377.911136:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573377.911140:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573377.911141:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573377.911144:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5290 00000100:00080000:2.0:1788573377.911147:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573377.911149:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573377.911157:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573377.911166:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627100 x1875449616347520/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573377.911179:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573377.911182:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573377.911184:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573377.911187:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5290 00000100:00080000:2.0:1788573377.911189:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573377.911191:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573377.911195:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573377.911200:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616347392:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788573377.911207:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616347648/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573377.911266:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573377.911302:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573377/real 1788573377] req@ffff9ada60624700 x1875449616347392/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573402 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573377.911320:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624700 x1875449616347392/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573402 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573377.911331:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573377.911335:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573377.911340:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616347392:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573377.911348:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616347520:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573377.911372:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573377.911383:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573377/real 1788573377] req@ffff9ada60627100 x1875449616347520/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573402 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573377.911391:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627100 x1875449616347520/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573402 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573377.911397:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573377.911399:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573377.911401:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616347520:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573377.911421:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616347648:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573377.911433:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573377.911440:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573377/real 1788573377] req@ffff9ada60626680 x1875449616347648/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573402 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573377.911448:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616347648/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573402 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573377.911453:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573377.911455:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573377.911457:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616347648:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573378.103023:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573378.423101:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573379.126932:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573379.447027:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573380.150960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573380.471041:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573381.174972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573381.494996:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573382.198965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573382.519057:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573383.030914:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573383.030922:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573383.030926:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573383.030934:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5295 00000100:00080000:2.0:1788573383.030944:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573383.030950:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573383.030981:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616347776/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573383.031004:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573383.031007:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573383.031009:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573383.031012:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5295 00000100:00080000:2.0:1788573383.031014:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573383.031016:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573383.031024:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573383.031033:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616347904/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573383.031040:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573383.031043:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573383.031044:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573383.031046:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5295 00000100:00080000:2.0:1788573383.031048:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573383.031050:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573383.031053:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573383.031060:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616348032/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573383.031068:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616347776:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573383.031164:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616347904:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573383.031211:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616348032:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573383.222957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573383.542976:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573384.246966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573384.567027:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573385.270985:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573385.591065:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1788573386.103117:0:2370:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:2.0:1788573386.103154:0:2370:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1788573386.103167:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788573386.103209:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788573386.103216:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:1.0:1788573386.103257:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573383/real 1788573386] req@ffff9ada60626a00 x1875449616348032/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573408 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573386.103279:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616348032/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573408 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573386.103288:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573386.103292:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573386.103295:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616348032:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573386.103301:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573383/real 1788573386] req@ffff9ada60625500 x1875449616347776/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573408 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573386.103310:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616347776/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573408 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573386.103314:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573386.103316:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573386.103318:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616347776:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573386.103323:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573383/real 1788573386] req@ffff9ada60624380 x1875449616347904/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573408 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573386.103329:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616347904/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573408 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573386.103334:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573386.103337:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573386.103339:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616347904:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573386.294923:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573386.614994:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573387.318956:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573387.639016:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573388.151008:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573388.151016:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573388.151020:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573388.151027:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5300 00000100:00080000:2.0:1788573388.151036:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573388.151043:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573388.151078:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616348160/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573388.151110:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573388.151113:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573388.151114:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573388.151117:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5300 00000100:00080000:2.0:1788573388.151120:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573388.151123:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573388.151130:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573388.151141:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616348288/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573388.151151:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573388.151155:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573388.151157:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573388.151162:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5300 00000100:00100000:1.0:1788573388.151164:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616348160:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788573388.151166:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573388.151175:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573388.151182:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573388.151196:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616348416/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573388.151252:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573388.151290:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573388/real 1788573388] req@ffff9ada60624a80 x1875449616348160/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573413 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573388.151310:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616348160/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573413 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573388.151321:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573388.151325:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573388.151329:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616348160:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573388.151337:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616348288:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573388.151361:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573388.151372:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573388/real 1788573388] req@ffff9ada60625880 x1875449616348288/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573413 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573388.151384:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616348288/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573413 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573388.151393:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573388.151396:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573388.151400:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616348288:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573388.151426:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616348416:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573388.151449:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573388.151463:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573388/real 1788573388] req@ffff9ada60625c00 x1875449616348416/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573413 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573388.151477:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616348416/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573413 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573388.151486:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573388.151489:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573388.151493:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616348416:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573388.342909:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573388.663024:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573389.366955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573389.687032:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573390.390922:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573390.711063:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573391.414954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573391.735019:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573392.438950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573392.759043:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573393.271030:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573393.271040:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573393.271043:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573393.271052:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5305 00000100:00080000:0.0:1788573393.271061:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573393.271067:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573393.271109:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616348544/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573393.271130:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573393.271133:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573393.271134:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573393.271137:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5305 00000100:00080000:0.0:1788573393.271139:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573393.271142:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573393.271150:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573393.271177:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616348672/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573393.271187:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573393.271190:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573393.271192:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573393.271196:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5305 00000100:00080000:0.0:1788573393.271200:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573393.271204:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573393.271210:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573393.271239:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c000 x1875449616348800/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573393.271255:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616348544:192.168.202.113@tcp:250:kworker.0 00000800:00000100:1.0:1788573393.271311:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573393.271360:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573393/real 1788573393] req@ffff9ada4557f480 x1875449616348544/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573418 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573393.271374:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616348544/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573418 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573393.271397:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573393.271400:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573393.271403:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616348544:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573393.271408:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616348672:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573393.271423:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573393.271430:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573393/real 1788573393] req@ffff9ada4557c700 x1875449616348672/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573418 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573393.271438:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616348672/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573418 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573393.271443:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573393.271444:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573393.271446:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616348672:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573393.271465:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616348800:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573393.271477:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573393.271483:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573393/real 1788573393] req@ffff9ada4557c000 x1875449616348800/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573418 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573393.271502:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c000 x1875449616348800/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573418 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573393.271507:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573393.271509:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573393.271511:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616348800:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573393.462979:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573393.783009:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573394.486955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573394.807129:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573395.510965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573395.831067:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573396.534936:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573396.855045:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573397.559033:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573397.879061:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573398.390970:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573398.390980:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573398.390985:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573398.390996:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5310 00000100:00080000:2.0:1788573398.391008:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573398.391016:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573398.391060:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616348928/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573398.391089:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573398.391094:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573398.391096:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573398.391100:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5310 00000100:00080000:2.0:1788573398.391104:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573398.391108:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573398.391122:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573398.391141:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616348928:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788573398.391142:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616349056/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573398.391158:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573398.391161:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573398.391162:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573398.391166:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5310 00000100:00080000:2.0:1788573398.391170:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573398.391173:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573398.391186:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573398.391197:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616349184/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573398.391235:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573398.391277:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573398/real 1788573398] req@ffff9ada60625f80 x1875449616348928/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573423 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573398.391293:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616348928/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573423 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573398.391302:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573398.391306:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573398.391315:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616348928:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573398.391340:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616349056:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573398.391357:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573398.391366:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573398/real 1788573398] req@ffff9ada60624e00 x1875449616349056/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573423 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573398.391375:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616349056/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573423 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573398.391380:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573398.391383:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573398.391385:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616349056:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573398.391389:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616349184:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573398.391399:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573398.391405:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573398/real 1788573398] req@ffff9ada60627480 x1875449616349184/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573423 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573398.391413:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616349184/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573423 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573398.391418:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573398.391419:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573398.391421:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616349184:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573398.583009:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573398.903044:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573399.606950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573399.927101:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573400.630992:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573400.951063:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573401.654973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573401.975109:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573402.678962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573402.999051:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573403.511006:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573403.511015:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573403.511019:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573403.511027:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5316 00000100:00080000:0.0:1788573403.511036:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573403.511042:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573403.511074:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c380 x1875449616349312/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573403.511096:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573403.511099:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573403.511101:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573403.511103:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5316 00000100:00080000:0.0:1788573403.511105:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573403.511108:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573403.511115:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573403.511122:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ce00 x1875449616349440/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573403.511129:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573403.511131:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573403.511132:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573403.511134:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5316 00000100:00080000:0.0:1788573403.511136:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573403.511138:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573403.511140:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573403.511147:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c000 x1875449616349568/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573403.511241:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616349312:192.168.202.113@tcp:250:kworker.0 00000800:00000100:1.0:1788573403.511323:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573403.511367:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573403/real 1788573403] req@ffff9ada4557c380 x1875449616349312/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573428 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573403.511384:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c380 x1875449616349312/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573428 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573403.511394:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573403.511398:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573403.511401:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616349312:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573403.511407:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616349440:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573403.511423:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573403.511431:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573403/real 1788573403] req@ffff9ada4557ce00 x1875449616349440/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573428 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573403.511440:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ce00 x1875449616349440/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573428 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573403.511445:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573403.511448:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573403.511450:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616349440:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573403.511454:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616349568:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573403.511465:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573403.511470:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573403/real 1788573403] req@ffff9ada4557c000 x1875449616349568/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573428 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573403.511479:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c000 x1875449616349568/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573428 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573403.511484:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573403.511486:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573403.511488:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616349568:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573403.702997:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573404.023057:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573404.726975:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573405.047028:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573405.750938:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573406.071011:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573406.775001:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573407.095017:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573407.798967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573408.119058:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573408.630958:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573408.630969:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573408.630974:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573408.630985:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5321 00000100:00080000:0.0:1788573408.630996:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573408.631005:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573408.631042:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616349696/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573408.631068:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573408.631071:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573408.631072:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573408.631075:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5321 00000100:00080000:0.0:1788573408.631078:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573408.631080:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573408.631088:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573408.631097:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616349824/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573408.631105:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573408.631108:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573408.631109:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573408.631111:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5321 00000100:00080000:0.0:1788573408.631114:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573408.631117:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573408.631121:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573408.631128:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616349952/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573408.631150:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616349696:192.168.202.113@tcp:250:kworker.0 00000800:00000100:1.0:1788573408.631243:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573408.631291:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573408/real 1788573408] req@ffff9ada4557c700 x1875449616349696/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573433 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573408.631309:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616349696/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573433 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573408.631319:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573408.631323:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573408.631326:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616349696:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573408.631332:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616349824:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573408.631350:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573408.631359:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573408/real 1788573408] req@ffff9ada4557f480 x1875449616349824/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573433 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573408.631375:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616349824/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573433 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573408.631381:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573408.631383:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573408.631385:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616349824:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573408.631410:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616349952:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573408.631423:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573408.631430:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573408/real 1788573408] req@ffff9ada4557ca80 x1875449616349952/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573433 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573408.631438:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616349952/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573433 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573408.631444:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573408.631447:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573408.631450:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616349952:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573408.823023:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573409.143056:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573409.846989:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573410.167034:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573410.870979:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573411.190991:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573411.894979:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573412.215016:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573412.919014:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573413.239002:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573413.750974:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573413.750985:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573413.750990:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573413.750998:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5326 00000100:00080000:0.0:1788573413.751007:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573413.751013:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573413.751052:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616350080/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573413.751075:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573413.751078:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573413.751080:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573413.751082:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5326 00000100:00080000:0.0:1788573413.751085:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573413.751087:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573413.751093:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573413.751101:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616350208/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573413.751108:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573413.751111:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573413.751113:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573413.751115:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5326 00000100:00080000:0.0:1788573413.751118:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573413.751121:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573413.751124:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573413.751135:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616350336/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573413.751136:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616350080:192.168.202.113@tcp:250:kworker.0 00000800:00000100:1.0:1788573413.751206:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573413.751237:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573413/real 1788573413] req@ffff9ada4557f800 x1875449616350080/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573438 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573413.751251:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616350080/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573438 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573413.751259:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573413.751268:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573413.751271:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616350080:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573413.751276:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616350208:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573413.751293:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573413.751301:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573413/real 1788573413] req@ffff9ada4557d880 x1875449616350208/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573438 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573413.751309:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616350208/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573438 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573413.751315:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573413.751317:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573413.751319:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616350208:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573413.751337:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616350336:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573413.751347:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573413.751353:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573413/real 1788573413] req@ffff9ada4557fb80 x1875449616350336/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573438 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573413.751361:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616350336/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573438 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573413.751371:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573413.751373:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573413.751375:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616350336:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573413.942963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573414.263006:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573414.966965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573415.286956:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573415.990948:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573416.311171:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573417.014988:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573417.335055:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573418.038930:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573418.359013:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573418.870977:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573418.870986:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573418.870989:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573418.870998:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5331 00000100:00080000:3.0:1788573418.871007:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573418.871013:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573418.871045:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2a00 x1875449616350464/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573418.871086:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573418.871090:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573418.871091:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573418.871094:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5331 00000100:00080000:3.0:1788573418.871097:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573418.871100:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573418.871107:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573418.871117:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0e00 x1875449616350592/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573418.871125:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573418.871128:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573418.871129:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573418.871131:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5331 00000100:00080000:3.0:1788573418.871134:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573418.871136:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573418.871139:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573418.871145:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1500 x1875449616350720/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573418.871158:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616350464:192.168.202.113@tcp:250:kworker.0 00000800:00000100:1.0:1788573418.871219:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573418.871250:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573418/real 1788573418] req@ffff9ada604e2a00 x1875449616350464/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573443 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573418.871265:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2a00 x1875449616350464/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573443 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573418.871273:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573418.871284:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573418.871293:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616350464:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573418.871301:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616350592:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573418.871326:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573418.871341:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573418/real 1788573418] req@ffff9ada604e0e00 x1875449616350592/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573443 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573418.871356:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0e00 x1875449616350592/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573443 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573418.871366:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573418.871370:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573418.871373:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616350592:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573418.871397:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616350720:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573418.871413:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573418.871422:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573418/real 1788573418] req@ffff9ada604e1500 x1875449616350720/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573443 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573418.871430:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1500 x1875449616350720/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573443 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573418.871435:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573418.871438:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573418.871440:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616350720:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573419.063014:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573419.383029:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573420.087026:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573420.407024:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573421.110954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573421.431049:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573422.134960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573422.454986:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573423.158947:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573423.479131:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573423.990939:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573423.990947:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573423.990950:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573423.990959:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5336 00000100:00080000:2.0:1788573423.990968:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573423.990974:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573423.991006:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616350848/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573423.991027:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573423.991030:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573423.991032:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573423.991034:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5336 00000100:00080000:2.0:1788573423.991037:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573423.991039:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573423.991046:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573423.991054:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616350976/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573423.991061:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573423.991064:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573423.991066:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573423.991069:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5336 00000100:00080000:2.0:1788573423.991072:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573423.991075:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573423.991078:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573423.991084:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616351104/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573423.991091:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616350848:192.168.202.113@tcp:250:kworker.0 00000800:00000100:1.0:1788573423.991165:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573423.991205:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573423/real 1788573423] req@ffff9ada60625180 x1875449616350848/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573448 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573423.991224:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616350848/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573448 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573423.991235:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573423.991239:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573423.991243:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616350848:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573423.991251:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616350976:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573423.991277:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573423.991299:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573423/real 1788573423] req@ffff9ada60626d80 x1875449616350976/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573448 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573423.991308:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616350976/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573448 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573423.991314:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573423.991316:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573423.991318:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616350976:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573423.991344:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616351104:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573423.991359:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573423.991367:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573423/real 1788573423] req@ffff9ada60627b80 x1875449616351104/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573448 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573423.991375:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616351104/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573448 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573423.991385:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573423.991387:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573423.991389:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616351104:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573424.182954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573424.502985:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573425.206956:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573425.527026:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573426.230954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573426.551029:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573427.254926:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573427.575021:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573428.278944:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573428.598966:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573429.110954:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573429.110963:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573429.110966:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573429.110974:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5341 00000100:00080000:0.0:1788573429.110983:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573429.110990:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573429.111020:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616351232/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573429.111042:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573429.111045:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573429.111047:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573429.111049:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5341 00000100:00080000:0.0:1788573429.111051:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573429.111054:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573429.111061:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573429.111070:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616351360/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573429.111077:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573429.111079:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573429.111081:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573429.111083:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5341 00000100:00080000:0.0:1788573429.111086:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1788573429.111091:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616351232:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788573429.111092:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573429.111099:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573429.111106:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616351488/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573429.111161:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573429.111191:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573429/real 1788573429] req@ffff9ada4557dc00 x1875449616351232/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573454 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573429.111204:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616351232/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573454 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573429.111212:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573429.111215:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573429.111218:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616351232:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573429.111223:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616351360:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573429.111238:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573429.111259:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573429/real 1788573429] req@ffff9ada4557e300 x1875449616351360/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573454 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573429.111274:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616351360/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573454 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573429.111280:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573429.111282:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573429.111284:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616351360:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573429.111308:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616351488:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573429.111330:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573429.111343:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573429/real 1788573429] req@ffff9ada4557ed80 x1875449616351488/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573454 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573429.111355:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616351488/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573454 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573429.111361:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573429.111363:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573429.111365:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616351488:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573429.302948:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573429.623008:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573430.326957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573430.647028:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573431.350942:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573431.671044:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573432.374969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573432.695015:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573433.398944:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573433.719025:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573434.231025:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573434.231035:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573434.231039:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573434.231049:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5346 00000100:00080000:3.0:1788573434.231059:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573434.231066:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573434.231103:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2680 x1875449616351616/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573434.231133:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573434.231137:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573434.231139:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573434.231143:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5346 00000100:00080000:3.0:1788573434.231146:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573434.231150:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573434.231160:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573434.231175:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1f80 x1875449616351744/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573434.231182:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616351616:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788573434.231200:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573434.231205:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573434.231207:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573434.231212:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5346 00000100:00080000:3.0:1788573434.231217:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573434.231222:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573434.231227:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573434.231241:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3100 x1875449616351872/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573434.231245:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573434.231280:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573434/real 1788573434] req@ffff9ada604e2680 x1875449616351616/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573459 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573434.231295:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2680 x1875449616351616/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573459 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573434.231303:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573434.231306:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573434.231309:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616351616:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573434.231336:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616351744:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573434.231369:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573434.231378:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573434/real 1788573434] req@ffff9ada604e1f80 x1875449616351744/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573459 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573434.231386:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1f80 x1875449616351744/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573459 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573434.231392:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573434.231394:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573434.231396:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616351744:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573434.231400:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616351872:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573434.231410:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573434.231416:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573434/real 1788573434] req@ffff9ada604e3100 x1875449616351872/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573459 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573434.231428:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3100 x1875449616351872/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573459 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573434.231433:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573434.231435:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573434.231436:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616351872:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573434.422956:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573434.742978:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573435.446948:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573435.767024:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573436.470965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573436.790980:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573437.494934:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573437.815047:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573438.518952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573438.839012:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788573439.350952:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573439.350961:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788573439.350964:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573439.350972:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5351 00000100:00080000:1.0:1788573439.350981:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573439.350987:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788573439.351020:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f100 x1875449616352000/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573439.351042:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573439.351045:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788573439.351046:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573439.351049:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5351 00000100:00080000:1.0:1788573439.351052:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573439.351055:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573439.351062:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573439.351072:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c380 x1875449616352128/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573439.351080:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573439.351083:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1788573439.351083:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616352000:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788573439.351085:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573439.351091:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5351 00000100:00080000:1.0:1788573439.351095:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573439.351098:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573439.351101:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573439.351108:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c700 x1875449616352256/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573439.351164:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573439.351194:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573439/real 1788573439] req@ffff9ada4566f100 x1875449616352000/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573464 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573439.351209:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f100 x1875449616352000/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573464 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573439.351217:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573439.351220:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573439.351223:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616352000:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573439.351242:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616352128:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573439.351258:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573439.351266:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573439/real 1788573439] req@ffff9ada4566c380 x1875449616352128/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573464 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573439.351279:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c380 x1875449616352128/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573464 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573439.351284:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573439.351286:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573439.351288:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616352128:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573439.351292:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616352256:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573439.351303:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573439.351309:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573439/real 1788573439] req@ffff9ada4566c700 x1875449616352256/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573464 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573439.351316:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c700 x1875449616352256/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573464 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573439.351321:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573439.351323:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573439.351325:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616352256:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573439.542961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573439.862996:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573440.566978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573440.887134:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573441.590988:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573441.910998:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573442.614941:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573442.935043:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573443.638948:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573443.959117:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573444.471068:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573444.471078:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573444.471083:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573444.471093:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5357 00000100:00080000:3.0:1788573444.471103:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573444.471110:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573444.471146:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3b80 x1875449616352384/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573444.471181:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573444.471185:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573444.471187:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573444.471191:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5357 00000100:00080000:3.0:1788573444.471195:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573444.471199:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573444.471208:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573444.471220:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1c00 x1875449616352512/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573444.471233:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1788573444.471241:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616352384:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788573444.471248:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573444.471255:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573444.471261:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5357 00000100:00080000:3.0:1788573444.471265:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573444.471271:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573444.471277:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573444.471291:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0000 x1875449616352640/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573444.471328:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573444.471370:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573444/real 1788573444] req@ffff9ada604e3b80 x1875449616352384/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573469 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573444.471390:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3b80 x1875449616352384/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573469 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573444.471399:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573444.471403:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573444.471405:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616352384:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573444.471430:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616352512:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573444.471446:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573444.471455:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573444/real 1788573444] req@ffff9ada604e1c00 x1875449616352512/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573469 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573444.471463:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1c00 x1875449616352512/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573469 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573444.471469:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573444.471471:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573444.471473:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616352512:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573444.471477:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616352640:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573444.471488:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573444.471494:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573444/real 1788573444] req@ffff9ada604e0000 x1875449616352640/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573469 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573444.471502:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0000 x1875449616352640/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573469 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573444.471512:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573444.471514:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573444.471516:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616352640:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573444.662937:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573444.983105:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573445.686955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573446.007010:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573446.710955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573447.031029:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573447.734954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573448.054980:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573448.758960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573449.078982:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573449.590990:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573449.590998:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573449.591002:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573449.591011:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5362 00000100:00080000:0.0:1788573449.591020:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573449.591027:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573449.591064:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616352768/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573449.591093:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573449.591096:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573449.591098:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573449.591101:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5362 00000100:00080000:0.0:1788573449.591104:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573449.591107:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573449.591114:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573449.591125:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d180 x1875449616352896/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573449.591133:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573449.591136:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573449.591138:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573449.591140:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5362 00000100:00080000:0.0:1788573449.591143:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573449.591145:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573449.591148:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573449.591152:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616352768:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788573449.591156:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616353024/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573449.591269:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616352896:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573449.591318:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616353024:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573449.782966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573450.103016:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573450.806946:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573451.127045:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573451.830965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573452.150980:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1788573452.663123:0:2372:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:0.0:1788573452.663169:0:2372:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1788573452.663197:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:0.0:1788573452.663242:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:0.0:1788573452.663253:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:1.0:1788573452.663356:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573449/real 1788573452] req@ffff9ada4557ea00 x1875449616353024/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573474 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573452.663384:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616353024/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573474 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573452.663397:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573452.663403:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573452.663407:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616353024:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573452.663414:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573449/real 1788573452] req@ffff9ada4557e680 x1875449616352768/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573474 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573452.663428:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616352768/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573474 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573452.663435:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573452.663439:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573452.663443:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616352768:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573452.663449:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573449/real 1788573452] req@ffff9ada4557d180 x1875449616352896/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573474 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573452.663460:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d180 x1875449616352896/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573474 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573452.663468:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573452.663471:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573452.663474:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616352896:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573452.854944:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573453.175039:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573453.878945:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573454.198992:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573454.710984:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573454.710993:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573454.710996:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573454.711006:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5367 00000100:00080000:2.0:1788573454.711016:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573454.711030:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573454.711067:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626300 x1875449616353152/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573454.711098:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573454.711103:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573454.711105:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573454.711109:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5367 00000100:00080000:2.0:1788573454.711114:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573454.711118:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573454.711128:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573454.711139:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627800 x1875449616353280/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573454.711148:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573454.711150:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573454.711151:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573454.711154:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5367 00000100:00080000:2.0:1788573454.711156:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573454.711159:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573454.711161:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573454.711169:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616353408/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573454.711446:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616353152:192.168.202.113@tcp:250:kworker.0 00000800:00000100:1.0:1788573454.711514:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573454.711548:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573454/real 1788573454] req@ffff9ada60626300 x1875449616353152/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573479 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573454.711567:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626300 x1875449616353152/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573479 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573454.711579:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573454.711584:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573454.711588:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616353152:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573454.711596:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616353280:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573454.711620:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573454.711635:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573454/real 1788573454] req@ffff9ada60627800 x1875449616353280/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573479 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573454.711648:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627800 x1875449616353280/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573479 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573454.711669:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573454.711674:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573454.711677:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616353280:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573454.711684:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616353408:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573454.711706:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573454.711734:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573454/real 1788573454] req@ffff9ada60627b80 x1875449616353408/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573479 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573454.711748:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616353408/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573479 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573454.711757:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573454.711761:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573454.711765:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616353408:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573454.902939:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573455.223032:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573455.926986:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573456.247086:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573456.950986:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573457.271044:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573457.974948:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573458.295053:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573458.998930:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573459.319095:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573459.830965:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573459.830973:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573459.830976:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573459.830984:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5372 00000100:00080000:0.0:1788573459.830994:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573459.831000:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573459.831030:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d500 x1875449616353536/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573459.831050:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573459.831053:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573459.831055:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573459.831058:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5372 00000100:00080000:0.0:1788573459.831060:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573459.831063:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573459.831071:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573459.831080:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557df80 x1875449616353664/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573459.831087:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573459.831090:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573459.831093:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1788573459.831094:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616353536:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788573459.831097:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5372 00000100:00080000:0.0:1788573459.831102:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573459.831105:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573459.831108:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573459.831117:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d180 x1875449616353792/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573459.831153:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573459.831181:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573459/real 1788573459] req@ffff9ada4557d500 x1875449616353536/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573484 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573459.831196:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d500 x1875449616353536/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573484 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573459.831203:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573459.831206:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573459.831209:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616353536:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573459.831226:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616353664:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573459.831240:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573459.831256:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573459/real 1788573459] req@ffff9ada4557df80 x1875449616353664/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573484 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573459.831265:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557df80 x1875449616353664/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573484 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573459.831270:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573459.831273:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573459.831275:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616353664:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573459.831278:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616353792:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573459.831290:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573459.831296:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573459/real 1788573459] req@ffff9ada4557d180 x1875449616353792/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573484 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573459.831304:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d180 x1875449616353792/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573484 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573459.831308:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573459.831310:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573459.831312:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616353792:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573460.022959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573460.343039:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573461.046976:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573461.367012:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573462.070939:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573462.390981:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573463.094974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573463.415006:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573464.118945:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573464.439055:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573464.950913:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573464.950924:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573464.950929:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573464.950939:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5377 00000100:00080000:0.0:1788573464.950950:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573464.950958:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573464.950992:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616353920/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573464.951016:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573464.951019:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573464.951020:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573464.951023:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5377 00000100:00080000:0.0:1788573464.951025:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573464.951028:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573464.951035:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573464.951043:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616354048/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573464.951051:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573464.951053:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573464.951055:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573464.951057:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5377 00000100:00080000:0.0:1788573464.951059:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573464.951061:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573464.951063:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573464.951069:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616354176/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573464.951075:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616353920:192.168.202.113@tcp:250:kworker.0 00000800:00000100:1.0:1788573464.951129:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573464.951160:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573464/real 1788573464] req@ffff9ada4557e680 x1875449616353920/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573489 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573464.951173:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616353920/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573489 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573464.951181:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573464.951184:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573464.951186:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616353920:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573464.951192:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616354048:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573464.951205:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573464.951217:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573464/real 1788573464] req@ffff9ada4557ea00 x1875449616354048/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573489 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573464.951229:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616354048/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573489 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573464.951237:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573464.951241:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573464.951245:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616354048:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573464.951270:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616354176:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573464.951291:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573464.951304:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573464/real 1788573464] req@ffff9ada4557ed80 x1875449616354176/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573489 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573464.951317:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616354176/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573489 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573464.951326:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573464.951328:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573464.951330:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616354176:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573465.142961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573465.463030:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573466.166949:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573466.487402:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573467.191023:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573467.510998:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573468.214953:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573468.535087:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573469.238971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573469.558987:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573470.070947:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573470.070959:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573470.070964:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573470.070976:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5382 00000100:00080000:0.0:1788573470.070988:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573470.070996:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573470.071038:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616354304/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573470.071065:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573470.071068:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573470.071070:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573470.071073:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5382 00000100:00080000:0.0:1788573470.071075:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573470.071077:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573470.071087:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573470.071098:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616354432/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573470.071111:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573470.071117:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:1.0:1788573470.071119:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616354304:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788573470.071120:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573470.071130:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5382 00000100:00080000:0.0:1788573470.071136:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573470.071140:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573470.071147:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573470.071160:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616354560/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573470.071198:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573470.071239:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573470/real 1788573470] req@ffff9ada4557e300 x1875449616354304/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573495 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573470.071264:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616354304/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573495 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573470.071274:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573470.071277:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573470.071280:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616354304:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573470.071304:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616354432:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573470.071321:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573470.071330:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573470/real 1788573470] req@ffff9ada4557dc00 x1875449616354432/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573495 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573470.071338:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616354432/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573495 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573470.071344:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573470.071347:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573470.071349:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616354432:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573470.071353:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616354560:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573470.071364:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573470.071370:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573470/real 1788573470] req@ffff9ada4557fb80 x1875449616354560/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573495 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573470.071383:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616354560/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573495 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573470.071388:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573470.071390:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573470.071392:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616354560:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573470.262967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573470.583051:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573471.286970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573471.606990:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573472.310998:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573472.631087:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573473.334981:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573473.655120:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573474.358940:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573474.679060:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573475.190955:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573475.190966:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573475.190970:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573475.190982:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5387 00000100:00080000:0.0:1788573475.190994:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573475.191002:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573475.191045:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616354688/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573475.191071:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573475.191074:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573475.191076:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573475.191078:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5387 00000100:00080000:0.0:1788573475.191081:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573475.191083:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573475.191092:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573475.191100:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616354816/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573475.191108:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573475.191111:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573475.191113:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573475.191115:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5387 00000100:00080000:0.0:1788573475.191120:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573475.191123:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573475.191126:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573475.191136:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616354944/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573475.191136:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616354688:192.168.202.113@tcp:250:kworker.0 00000800:00000100:1.0:1788573475.191219:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573475.191260:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573475/real 1788573475] req@ffff9ada4557d880 x1875449616354688/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573500 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573475.191276:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616354688/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573500 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573475.191286:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573475.191289:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573475.191292:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616354688:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573475.191297:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616354816:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573475.191312:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573475.191320:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573475/real 1788573475] req@ffff9ada4557f800 x1875449616354816/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573500 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573475.191329:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616354816/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573500 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573475.191335:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573475.191336:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573475.191338:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616354816:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573475.191360:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616354944:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573475.191371:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573475.191378:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573475/real 1788573475] req@ffff9ada4557ca80 x1875449616354944/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573500 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573475.191386:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616354944/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573500 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573475.191391:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573475.191393:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573475.191394:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616354944:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573475.383004:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573475.702972:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573476.406933:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573476.726981:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573477.430954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573477.751031:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573478.454969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573478.775008:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573479.478963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573479.798980:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573480.310973:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573480.310980:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573480.310984:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573480.310994:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5392 00000100:00080000:2.0:1788573480.311005:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573480.311013:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573480.311053:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616355072/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573480.311081:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573480.311086:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573480.311087:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573480.311092:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5392 00000100:00080000:2.0:1788573480.311095:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573480.311100:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573480.311111:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573480.311125:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616355200/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573480.311135:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573480.311139:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573480.311141:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573480.311145:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5392 00000100:00080000:2.0:1788573480.311149:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1788573480.311156:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616355072:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788573480.311157:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573480.311169:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573480.311186:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616355328/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573480.311232:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573480.311268:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573480/real 1788573480] req@ffff9ada60626d80 x1875449616355072/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573505 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573480.311288:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616355072/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573505 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573480.311300:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573480.311305:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573480.311309:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616355072:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573480.311318:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616355200:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573480.311343:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573480.311357:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573480/real 1788573480] req@ffff9ada60625180 x1875449616355200/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573505 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573480.311369:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616355200/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573505 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573480.311379:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573480.311382:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573480.311386:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616355200:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573480.311413:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616355328:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573480.311435:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573480.311446:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573480/real 1788573480] req@ffff9ada60627480 x1875449616355328/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573505 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573480.311459:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616355328/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573505 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573480.311476:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573480.311480:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573480.311484:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616355328:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573480.502976:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573480.823030:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573481.526967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573481.847053:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573482.550986:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573482.871094:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573483.574940:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573483.895093:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573484.599032:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573484.919095:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573485.431040:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573485.431051:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573485.431056:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573485.431065:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5397 00000100:00080000:3.0:1788573485.431075:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573485.431083:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573485.431124:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0a80 x1875449616355456/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573485.431155:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573485.431159:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573485.431162:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573485.431166:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5397 00000100:00080000:3.0:1788573485.431170:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573485.431174:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573485.431184:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573485.431197:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2d80 x1875449616355584/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573485.431207:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616355456:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788573485.431207:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573485.431216:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573485.431219:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573485.431222:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5397 00000100:00080000:3.0:1788573485.431225:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573485.431228:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573485.431231:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573485.431240:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2300 x1875449616355712/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573485.431272:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573485.431306:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573485/real 1788573485] req@ffff9ada604e0a80 x1875449616355456/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573510 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573485.431326:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0a80 x1875449616355456/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573510 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573485.431338:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573485.431342:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573485.431347:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616355456:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573485.431368:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616355584:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573485.431386:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573485.431396:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573485/real 1788573485] req@ffff9ada604e2d80 x1875449616355584/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573510 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573485.431404:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2d80 x1875449616355584/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573510 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573485.431410:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573485.431412:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573485.431415:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616355584:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573485.431418:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616355712:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573485.431430:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573485.431436:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573485/real 1788573485] req@ffff9ada604e2300 x1875449616355712/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573510 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573485.431443:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2300 x1875449616355712/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573510 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573485.431448:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573485.431450:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573485.431452:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616355712:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573485.622959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573485.943074:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573486.646969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573486.967127:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573487.670969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573487.991125:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573488.695007:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573489.015034:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573489.719012:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573490.039065:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573490.550986:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573490.551004:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573490.551008:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573490.551019:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5403 00000100:00080000:3.0:1788573490.551031:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573490.551039:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573490.551089:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1180 x1875449616355840/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573490.551125:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573490.551128:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573490.551130:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573490.551133:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5403 00000100:00080000:3.0:1788573490.551136:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573490.551139:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573490.551151:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573490.551163:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1880 x1875449616355968/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573490.551174:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573490.551177:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573490.551178:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573490.551182:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5403 00000100:00100000:1.0:1788573490.551186:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616355840:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788573490.551186:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573490.551193:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573490.551198:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573490.551205:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3800 x1875449616356096/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573490.551267:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573490.551313:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573490/real 1788573490] req@ffff9ada604e1180 x1875449616355840/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573515 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573490.551330:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1180 x1875449616355840/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573515 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573490.551341:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573490.551344:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573490.551347:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616355840:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573490.551371:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616355968:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573490.551390:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573490.551399:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573490/real 1788573490] req@ffff9ada604e1880 x1875449616355968/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573515 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573490.551407:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1880 x1875449616355968/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573515 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573490.551413:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573490.551416:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573490.551418:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616355968:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573490.551421:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616356096:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573490.551433:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573490.551439:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573490/real 1788573490] req@ffff9ada604e3800 x1875449616356096/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573515 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573490.551446:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3800 x1875449616356096/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573515 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573490.551452:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573490.551454:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573490.551455:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616356096:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573490.742987:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573491.062976:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573491.767007:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573492.087132:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573492.790988:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573493.111011:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573493.814955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573494.135009:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573494.838975:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573495.159050:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573495.670991:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573495.671000:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573495.671003:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573495.671012:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5408 00000100:00080000:2.0:1788573495.671020:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573495.671026:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573495.671060:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616356224/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573495.671083:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573495.671085:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573495.671087:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573495.671089:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5408 00000100:00080000:2.0:1788573495.671092:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573495.671095:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573495.671102:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573495.671110:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616356352/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573495.671117:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573495.671119:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573495.671120:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573495.671122:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5408 00000100:00080000:2.0:1788573495.671124:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573495.671126:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573495.671129:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573495.671136:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616356480/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573495.671168:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616356224:192.168.202.113@tcp:250:kworker.0 00000800:00000100:1.0:1788573495.671223:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573495.671252:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573495/real 1788573495] req@ffff9ada60624e00 x1875449616356224/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573520 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573495.671266:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616356224/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573520 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573495.671273:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573495.671276:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573495.671279:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616356224:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573495.671284:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616356352:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573495.671297:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573495.671305:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573495/real 1788573495] req@ffff9ada60625f80 x1875449616356352/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573520 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573495.671312:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616356352/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573520 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573495.671317:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573495.671319:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573495.671321:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616356352:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573495.671325:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616356480:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573495.671334:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573495.671340:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573495/real 1788573495] req@ffff9ada60625c00 x1875449616356480/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573520 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573495.671347:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616356480/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573520 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573495.671353:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573495.671354:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573495.671356:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616356480:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573495.862928:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573496.183036:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573496.886931:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573497.207087:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573497.910930:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573498.230969:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573498.934954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573499.255070:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573499.958960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573500.279000:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573500.790985:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573500.790996:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573500.791000:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573500.791011:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5413 00000100:00080000:2.0:1788573500.791022:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573500.791030:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573500.791068:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616356608/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573500.791099:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573500.791104:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573500.791106:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573500.791110:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5413 00000100:00080000:2.0:1788573500.791114:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573500.791118:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573500.791126:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573500.791139:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616356736/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573500.791149:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573500.791153:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573500.791155:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573500.791159:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5413 00000100:00080000:2.0:1788573500.791163:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1788573500.791171:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616356608:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788573500.791171:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573500.791181:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573500.791201:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616356864/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573500.791230:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573500.791262:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573500/real 1788573500] req@ffff9ada60625880 x1875449616356608/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573525 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573500.791277:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616356608/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573525 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573500.791285:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573500.791288:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573500.791291:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616356608:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573500.791297:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616356736:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573500.791311:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573500.791319:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573500/real 1788573500] req@ffff9ada60624a80 x1875449616356736/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573525 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573500.791334:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616356736/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573525 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573500.791340:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573500.791342:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573500.791345:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616356736:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573500.791362:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616356864:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573500.791374:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573500.791381:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573500/real 1788573500] req@ffff9ada60624380 x1875449616356864/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573525 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573500.791389:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616356864/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573525 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573500.791394:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573500.791396:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573500.791398:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616356864:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573500.982938:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573501.303027:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573502.006957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573502.327025:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573503.030926:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573503.351046:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573504.054985:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573504.375024:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573505.078944:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573505.399094:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573505.910990:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573505.911001:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573505.911005:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573505.911019:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5418 00000100:00080000:2.0:1788573505.911034:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573505.911045:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573505.911103:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616356992/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573505.911135:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573505.911138:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573505.911140:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573505.911147:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5418 00000100:00080000:2.0:1788573505.911150:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573505.911153:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573505.911164:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573505.911174:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616357120/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573505.911182:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573505.911185:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573505.911186:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573505.911189:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5418 00000100:00080000:2.0:1788573505.911191:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573505.911197:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788573505.911200:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616356992:192.168.202.113@tcp:250:kworker.0 00000008:00100000:2.0:1788573505.911200:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573505.911210:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616357248/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573505.911278:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573505.911318:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573505/real 1788573505] req@ffff9ada60625500 x1875449616356992/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573530 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573505.911335:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616356992/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573530 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573505.911344:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573505.911347:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573505.911350:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616356992:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573505.911356:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616357120:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573505.911373:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573505.911381:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573505/real 1788573505] req@ffff9ada60626a00 x1875449616357120/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573530 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573505.911389:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616357120/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573530 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573505.911396:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573505.911398:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573505.911401:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616357120:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573505.911426:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616357248:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573505.911447:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573505.911459:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573505/real 1788573505] req@ffff9ada60626680 x1875449616357248/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573530 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573505.911473:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616357248/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573530 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573505.911480:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573505.911482:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573505.911484:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616357248:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573506.102973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573506.423082:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573507.126956:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573507.447089:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573508.150956:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573508.471012:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573509.174975:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573509.495407:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573510.198971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573510.518999:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573511.031019:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573511.031031:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573511.031036:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573511.031049:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5423 00000100:00080000:0.0:1788573511.031060:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573511.031069:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573511.031106:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616357376/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573511.031133:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573511.031136:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573511.031138:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573511.031141:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5423 00000100:00080000:0.0:1788573511.031144:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573511.031147:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573511.031155:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573511.031164:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616357504/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573511.031172:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573511.031175:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573511.031176:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573511.031179:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5423 00000100:00080000:0.0:1788573511.031181:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573511.031184:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573511.031187:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573511.031206:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c000 x1875449616357632/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573511.031207:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616357376:192.168.202.113@tcp:250:kworker.0 00000800:00000100:1.0:1788573511.031272:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573511.031305:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573511/real 1788573511] req@ffff9ada4557f480 x1875449616357376/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573536 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573511.031319:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616357376/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573536 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573511.031328:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573511.031331:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573511.031334:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616357376:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573511.031339:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616357504:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573511.031376:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573511.031388:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573511/real 1788573511] req@ffff9ada4557c700 x1875449616357504/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573536 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573511.031401:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616357504/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573536 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573511.031409:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573511.031412:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573511.031415:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616357504:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573511.031441:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616357632:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573511.031458:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573511.031467:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573511/real 1788573511] req@ffff9ada4557c000 x1875449616357632/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573536 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573511.031479:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c000 x1875449616357632/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573536 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573511.031487:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573511.031490:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573511.031493:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616357632:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573511.222964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573511.543056:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573512.246954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573512.567051:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573513.270935:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573513.591033:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573514.294954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573514.615045:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573515.318927:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573515.638984:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573516.150965:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573516.150972:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573516.150976:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573516.150985:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5428 00000100:00080000:0.0:1788573516.150994:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573516.151000:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573516.151034:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ce00 x1875449616357760/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573516.151058:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573516.151060:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573516.151062:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573516.151064:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5428 00000100:00080000:0.0:1788573516.151067:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573516.151069:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573516.151076:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573516.151084:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c380 x1875449616357888/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573516.151091:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573516.151094:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573516.151096:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573516.151098:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5428 00000100:00080000:0.0:1788573516.151100:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573516.151103:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573516.151105:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573516.151113:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c000 x1875449616358016/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573516.151158:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616357760:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573516.151294:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616357888:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573516.151333:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616358016:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573516.342965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573516.663006:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573517.366982:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573517.687021:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573518.390951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573518.711063:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1788573519.223148:0:2373:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:2.0:1788573519.223197:0:2373:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1788573519.223220:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788573519.223276:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788573519.223288:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:1.0:1788573519.223368:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573516/real 1788573519] req@ffff9ada4557ce00 x1875449616357760/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573541 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573519.223419:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ce00 x1875449616357760/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573541 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573519.223437:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573519.223444:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573519.223449:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616357760:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573519.223460:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573516/real 1788573519] req@ffff9ada4557c380 x1875449616357888/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573541 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573519.223484:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c380 x1875449616357888/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573541 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573519.223495:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573519.223499:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573519.223504:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616357888:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573519.223512:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573516/real 1788573519] req@ffff9ada4557c000 x1875449616358016/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573541 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573519.223527:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c000 x1875449616358016/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573541 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573519.223540:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573519.223544:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573519.223548:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616358016:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573519.415034:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573519.735061:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573520.438970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573520.759090:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788573521.271015:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573521.271034:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788573521.271040:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573521.271060:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5433 00000100:00080000:1.0:1788573521.271076:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573521.271090:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788573521.271153:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ce00 x1875449616358144/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573521.271197:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573521.271203:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788573521.271206:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573521.271212:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5433 00000100:00080000:1.0:1788573521.271219:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573521.271226:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573521.271242:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573521.271265:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616358144:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573521.271269:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d880 x1875449616358272/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573521.271293:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573521.271298:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788573521.271301:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573521.271307:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5433 00000100:00080000:1.0:1788573521.271313:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573521.271319:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573521.271328:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573521.271347:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e680 x1875449616358400/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573521.271399:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573521.271459:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573521/real 1788573521] req@ffff9ada4566ce00 x1875449616358144/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573546 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573521.271488:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ce00 x1875449616358144/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573546 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573521.271502:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573521.271515:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573521.271522:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616358144:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573521.271557:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616358272:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573521.271608:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573521.271629:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573521/real 1788573521] req@ffff9ada4566d880 x1875449616358272/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573546 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573521.271681:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d880 x1875449616358272/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573546 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573521.271694:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573521.271699:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573521.271704:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616358272:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573521.271713:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616358400:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573521.271741:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573521.271758:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573521/real 1788573521] req@ffff9ada4566e680 x1875449616358400/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573546 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573521.271775:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e680 x1875449616358400/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573546 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573521.271785:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573521.271790:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573521.271794:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616358400:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573521.462974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573521.783017:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573522.486953:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573522.806992:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573523.510958:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573523.831157:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573524.534964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573524.855048:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573525.558970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573525.879030:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573526.390962:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573526.390970:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573526.390973:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573526.390982:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5438 00000100:00080000:3.0:1788573526.390991:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573526.390997:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573526.391030:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0380 x1875449616358528/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573526.391053:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573526.391056:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573526.391058:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573526.391060:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5438 00000100:00080000:3.0:1788573526.391063:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573526.391066:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573526.391075:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573526.391086:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616358528:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573526.391087:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0700 x1875449616358656/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573526.391100:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573526.391103:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573526.391104:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573526.391108:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5438 00000100:00080000:3.0:1788573526.391110:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573526.391112:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573526.391115:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573526.391125:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3480 x1875449616358784/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573526.391151:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573526.391181:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573526/real 1788573526] req@ffff9ada604e0380 x1875449616358528/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573551 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573526.391196:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0380 x1875449616358528/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573551 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573526.391204:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573526.391207:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573526.391210:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616358528:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573526.391228:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616358656:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573526.391243:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573526.391251:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573526/real 1788573526] req@ffff9ada604e0700 x1875449616358656/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573551 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573526.391259:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0700 x1875449616358656/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573551 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573526.391265:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573526.391267:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573526.391270:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616358656:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573526.391273:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616358784:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573526.391284:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573526.391290:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573526/real 1788573526] req@ffff9ada604e3480 x1875449616358784/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573551 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573526.391297:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3480 x1875449616358784/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573551 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573526.391302:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573526.391304:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573526.391306:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616358784:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573526.582938:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573526.903035:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573527.606958:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573527.927004:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573528.630939:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573528.951026:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573529.654957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573529.974977:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573530.679018:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573530.999076:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573531.510961:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573531.510976:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573531.510980:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573531.510989:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5444 00000100:00080000:3.0:1788573531.510998:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573531.511004:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573531.511038:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3480 x1875449616358912/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573531.511062:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573531.511065:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573531.511066:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573531.511069:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5444 00000100:00080000:3.0:1788573531.511072:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573531.511074:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573531.511081:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573531.511089:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0700 x1875449616359040/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573531.511097:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573531.511099:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573531.511101:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573531.511103:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5444 00000100:00080000:3.0:1788573531.511106:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573531.511109:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573531.511112:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573531.511118:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0380 x1875449616359168/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573531.511128:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616358912:192.168.202.113@tcp:250:kworker.0 00000800:00000100:1.0:1788573531.511196:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573531.511240:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573531/real 1788573531] req@ffff9ada604e3480 x1875449616358912/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573556 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573531.511255:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3480 x1875449616358912/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573556 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573531.511263:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573531.511266:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573531.511268:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616358912:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573531.511284:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616359040:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573531.511310:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573531.511324:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573531/real 1788573531] req@ffff9ada604e0700 x1875449616359040/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573556 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573531.511346:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0700 x1875449616359040/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573556 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573531.511352:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573531.511354:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573531.511357:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616359040:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573531.511376:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616359168:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573531.511392:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573531.511400:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573531/real 1788573531] req@ffff9ada604e0380 x1875449616359168/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573556 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573531.511408:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0380 x1875449616359168/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573556 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573531.511418:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573531.511420:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573531.511422:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616359168:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573531.702989:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573532.023048:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573532.726934:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573533.047028:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573533.750991:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573534.071106:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573534.774966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573535.095031:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573535.798987:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573536.119057:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573536.630921:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573536.630930:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573536.630934:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573536.630942:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5449 00000100:00080000:3.0:1788573536.630951:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573536.630958:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573536.630991:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3800 x1875449616359296/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573536.631015:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573536.631018:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573536.631020:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573536.631022:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5449 00000100:00080000:3.0:1788573536.631025:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573536.631027:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573536.631034:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573536.631042:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1880 x1875449616359424/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573536.631055:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573536.631058:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573536.631059:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573536.631062:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5449 00000100:00080000:3.0:1788573536.631065:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573536.631068:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573536.631070:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573536.631076:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1180 x1875449616359552/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573536.631107:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616359296:192.168.202.113@tcp:250:kworker.0 00000800:00000100:1.0:1788573536.631196:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573536.631238:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573536/real 1788573536] req@ffff9ada604e3800 x1875449616359296/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573561 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573536.631254:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3800 x1875449616359296/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573561 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573536.631264:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573536.631268:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573536.631271:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616359296:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573536.631278:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616359424:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573536.631295:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573536.631305:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573536/real 1788573536] req@ffff9ada604e1880 x1875449616359424/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573561 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573536.631313:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1880 x1875449616359424/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573561 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573536.631320:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573536.631322:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573536.631324:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616359424:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573536.631328:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616359552:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573536.631339:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573536.631345:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573536/real 1788573536] req@ffff9ada604e1180 x1875449616359552/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573561 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573536.631352:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1180 x1875449616359552/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573561 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573536.631358:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573536.631360:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573536.631361:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616359552:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573536.822984:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573537.142989:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573537.846962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573538.167023:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573538.871005:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573539.190976:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573539.894972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573540.215048:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573540.918957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573541.239028:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573541.750935:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573541.750944:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573541.750947:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573541.750955:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5454 00000100:00080000:3.0:1788573541.750964:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573541.750970:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573541.751004:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2300 x1875449616359680/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573541.751027:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573541.751030:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573541.751032:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573541.751034:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5454 00000100:00080000:3.0:1788573541.751036:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573541.751039:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573541.751048:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573541.751061:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2d80 x1875449616359808/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573541.751073:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573541.751077:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573541.751079:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573541.751085:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5454 00000100:00100000:1.0:1788573541.751087:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616359680:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788573541.751089:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573541.751096:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573541.751101:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573541.751110:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0a80 x1875449616359936/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573541.751167:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573541.751203:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573541/real 1788573541] req@ffff9ada604e2300 x1875449616359680/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573566 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573541.751224:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2300 x1875449616359680/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573566 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573541.751236:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573541.751241:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573541.751245:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616359680:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573541.751253:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616359808:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573541.751276:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573541.751288:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573541/real 1788573541] req@ffff9ada604e2d80 x1875449616359808/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573566 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573541.751302:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2d80 x1875449616359808/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573566 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573541.751311:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573541.751315:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573541.751319:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616359808:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573541.751342:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616359936:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573541.751362:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573541.751373:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573541/real 1788573541] req@ffff9ada604e0a80 x1875449616359936/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573566 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573541.751386:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0a80 x1875449616359936/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573566 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573541.751395:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573541.751399:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573541.751403:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616359936:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573541.942940:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573542.263067:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573542.966954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573543.287048:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573543.990982:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573544.311065:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573545.015038:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573545.335165:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573546.038963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573546.359032:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788573546.871016:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573546.871024:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788573546.871027:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573546.871036:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5459 00000100:00080000:1.0:1788573546.871046:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573546.871052:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788573546.871085:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e680 x1875449616360064/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573546.871107:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573546.871110:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788573546.871112:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573546.871114:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5459 00000100:00080000:1.0:1788573546.871117:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573546.871119:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573546.871126:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573546.871136:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d880 x1875449616360192/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573546.871144:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573546.871146:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788573546.871149:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1788573546.871153:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616360064:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788573546.871154:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5459 00000100:00080000:1.0:1788573546.871159:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573546.871165:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573546.871169:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573546.871179:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ce00 x1875449616360320/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573546.871225:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573546.871260:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573546/real 1788573546] req@ffff9ada4566e680 x1875449616360064/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573571 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573546.871281:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e680 x1875449616360064/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573571 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573546.871293:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573546.871297:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573546.871301:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616360064:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573546.871336:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616360192:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573546.871363:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573546.871377:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573546/real 1788573546] req@ffff9ada4566d880 x1875449616360192/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573571 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573546.871391:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d880 x1875449616360192/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573571 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573546.871400:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573546.871404:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573546.871408:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616360192:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573546.871414:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616360320:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573546.871436:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573546.871448:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573546/real 1788573546] req@ffff9ada4566ce00 x1875449616360320/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573571 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573546.871462:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ce00 x1875449616360320/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573571 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573546.871471:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573546.871475:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573546.871478:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616360320:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573547.062976:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573547.382992:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573548.086980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573548.406993:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573549.110936:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573549.430965:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573550.134957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573550.455074:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573551.158989:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573551.479018:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788573551.990962:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573551.990971:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788573551.990974:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573551.990983:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5464 00000100:00080000:1.0:1788573551.990992:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573551.990998:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788573551.991028:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c700 x1875449616360448/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573551.991052:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573551.991055:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788573551.991056:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573551.991059:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5464 00000100:00080000:1.0:1788573551.991061:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573551.991063:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573551.991069:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573551.991076:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c380 x1875449616360576/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573551.991083:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573551.991086:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788573551.991087:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573551.991090:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5464 00000100:00080000:1.0:1788573551.991092:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573551.991095:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573551.991097:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573551.991103:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f100 x1875449616360704/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788573551.991117:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616360448:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788573551.991178:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573551.991213:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573551/real 1788573551] req@ffff9ada4566c700 x1875449616360448/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573576 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573551.991241:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c700 x1875449616360448/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573576 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573551.991253:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573551.991257:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573551.991262:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616360448:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573551.991269:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616360576:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573551.991292:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573551.991304:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573551/real 1788573551] req@ffff9ada4566c380 x1875449616360576/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573576 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573551.991318:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c380 x1875449616360576/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573576 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573551.991328:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573551.991331:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573551.991335:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616360576:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573551.991360:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616360704:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573551.991382:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573551.991395:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573551/real 1788573551] req@ffff9ada4566f100 x1875449616360704/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573576 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573551.991409:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f100 x1875449616360704/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573576 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573551.991424:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573551.991428:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573551.991432:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616360704:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573552.182986:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573552.503071:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573553.207015:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573553.527034:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573554.230962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573554.551002:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573555.254983:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573555.575055:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573556.278955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573556.599056:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573557.111040:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573557.111055:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573557.111061:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573557.111075:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5469 00000100:00080000:3.0:1788573557.111091:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573557.111101:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573557.111155:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0000 x1875449616360832/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573557.111199:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573557.111204:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573557.111208:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573557.111213:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5469 00000100:00080000:3.0:1788573557.111218:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573557.111224:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573557.111238:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573557.111255:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1c00 x1875449616360960/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573557.111264:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616360832:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788573557.111272:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573557.111278:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573557.111281:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573557.111285:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5469 00000100:00080000:3.0:1788573557.111290:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573557.111295:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573557.111301:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573557.111317:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3b80 x1875449616361088/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573557.111377:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573557.111429:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573557/real 1788573557] req@ffff9ada604e0000 x1875449616360832/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573582 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573557.111452:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0000 x1875449616360832/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573582 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573557.111463:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573557.111466:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573557.111470:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616360832:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573557.111495:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616360960:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573557.111528:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573557.111538:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573557/real 1788573557] req@ffff9ada604e1c00 x1875449616360960/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573582 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573557.111547:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1c00 x1875449616360960/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573582 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573557.111554:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573557.111556:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573557.111559:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616360960:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573557.111563:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616361088:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573557.111576:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573557.111582:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573557/real 1788573557] req@ffff9ada604e3b80 x1875449616361088/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573582 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573557.111590:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3b80 x1875449616361088/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573582 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573557.111596:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573557.111599:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573557.111603:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616361088:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573557.302959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573557.623032:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573558.326953:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573558.647004:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573559.350935:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573559.671001:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573560.374961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573560.695029:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573561.398934:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573561.719023:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573562.230999:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573562.231007:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573562.231012:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573562.231020:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5474 00000100:00080000:2.0:1788573562.231029:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573562.231035:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573562.231070:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627100 x1875449616361216/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573562.231094:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573562.231097:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573562.231098:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573562.231101:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5474 00000100:00080000:2.0:1788573562.231103:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573562.231106:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573562.231114:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573562.231122:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624700 x1875449616361344/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573562.231128:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573562.231130:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573562.231131:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573562.231133:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5474 00000100:00080000:2.0:1788573562.231135:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573562.231138:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573562.231140:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573562.231147:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616361472/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573562.231167:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616361216:192.168.202.113@tcp:250:kworker.0 00000800:00000100:1.0:1788573562.231230:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573562.231265:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573562/real 1788573562] req@ffff9ada60627100 x1875449616361216/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573587 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573562.231282:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627100 x1875449616361216/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573587 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573562.231293:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573562.231297:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573562.231302:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616361216:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573562.231310:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616361344:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573562.231334:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573562.231345:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573562/real 1788573562] req@ffff9ada60624700 x1875449616361344/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573587 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573562.231354:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624700 x1875449616361344/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573587 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573562.231359:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573562.231362:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573562.231366:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616361344:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573562.231372:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616361472:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573562.231392:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573562.231403:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573562/real 1788573562] req@ffff9ada60626680 x1875449616361472/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573587 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573562.231414:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616361472/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573587 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573562.231421:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573562.231425:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573562.231428:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616361472:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573562.422955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573562.743056:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573563.446933:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573563.767054:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573564.470967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573564.791040:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573565.494942:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573565.815036:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573566.518981:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573566.839120:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573567.350962:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573567.350970:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573567.350974:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573567.350983:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5479 00000100:00080000:3.0:1788573567.350991:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573567.350997:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573567.351039:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3100 x1875449616361600/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573567.351061:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573567.351065:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573567.351067:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573567.351071:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5479 00000100:00080000:3.0:1788573567.351075:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573567.351080:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573567.351090:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573567.351105:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616361600:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573567.351106:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1f80 x1875449616361728/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573567.351128:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573567.351130:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573567.351131:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573567.351135:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5479 00000100:00080000:3.0:1788573567.351138:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573567.351140:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573567.351144:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573567.351155:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2680 x1875449616361856/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573567.351179:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573567.351216:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573567/real 1788573567] req@ffff9ada604e3100 x1875449616361600/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573592 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573567.351235:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3100 x1875449616361600/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573592 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573567.351246:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573567.351250:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573567.351255:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616361600:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573567.351280:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616361728:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573567.351305:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573567.351316:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573567/real 1788573567] req@ffff9ada604e1f80 x1875449616361728/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573592 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573567.351324:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1f80 x1875449616361728/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573592 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573567.351343:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573567.351346:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573567.351348:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616361728:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573567.351353:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616361856:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573567.351367:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573567.351374:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573567/real 1788573567] req@ffff9ada604e2680 x1875449616361856/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573592 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573567.351382:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2680 x1875449616361856/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573592 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573567.351387:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573567.351389:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573567.351391:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616361856:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573567.542946:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573567.863004:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573568.566977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573568.886989:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573569.590964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573569.911050:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573570.614964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573570.935020:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573571.638973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573571.959009:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573572.470980:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573572.470991:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573572.470996:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573572.471005:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5485 00000100:00080000:3.0:1788573572.471015:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573572.471023:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573572.471060:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1500 x1875449616361984/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573572.471090:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573572.471093:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573572.471095:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573572.471099:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5485 00000100:00080000:3.0:1788573572.471103:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573572.471107:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573572.471117:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573572.471133:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0e00 x1875449616362112/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573572.471144:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573572.471148:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573572.471151:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573572.471156:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5485 00000100:00080000:3.0:1788573572.471159:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573572.471163:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573572.471171:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573572.471173:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616361984:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573572.471186:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2a00 x1875449616362240/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573572.471269:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573572.471313:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573572/real 1788573572] req@ffff9ada604e1500 x1875449616361984/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573597 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573572.471331:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1500 x1875449616361984/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573597 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573572.471340:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573572.471344:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573572.471347:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616361984:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573572.471353:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616362112:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573572.471370:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573572.471379:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573572/real 1788573572] req@ffff9ada604e0e00 x1875449616362112/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573597 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573572.471393:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0e00 x1875449616362112/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573597 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573572.471403:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573572.471406:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573572.471411:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616362112:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573572.471442:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616362240:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573572.471460:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573572.471469:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573572/real 1788573572] req@ffff9ada604e2a00 x1875449616362240/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573597 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573572.471478:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2a00 x1875449616362240/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573597 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573572.471484:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573572.471486:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573572.471488:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616362240:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573572.662992:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573572.983041:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573573.686981:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573574.007065:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573574.711002:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573575.031103:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573575.734919:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573576.056831:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573576.758935:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573577.079061:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573577.590949:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573577.590958:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573577.590962:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573577.590971:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5490 00000100:00080000:0.0:1788573577.590980:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573577.590985:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573577.591020:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616362368/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573577.591063:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573577.591067:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573577.591069:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573577.591074:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5490 00000100:00080000:0.0:1788573577.591077:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573577.591081:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573577.591089:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573577.591102:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616362496/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573577.591110:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573577.591112:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573577.591114:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573577.591116:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5490 00000100:00080000:0.0:1788573577.591119:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573577.591121:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573577.591124:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573577.591134:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616362624/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573577.591140:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616362368:192.168.202.113@tcp:250:kworker.0 00000800:00000100:1.0:1788573577.591234:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573577.591285:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573577/real 1788573577] req@ffff9ada4557c700 x1875449616362368/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573602 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573577.591309:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616362368/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573602 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573577.591324:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573577.591329:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573577.591334:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616362368:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573577.591353:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616362496:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573577.591380:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573577.591394:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573577/real 1788573577] req@ffff9ada4557f480 x1875449616362496/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573602 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573577.591409:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616362496/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573602 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573577.591418:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573577.591422:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573577.591426:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616362496:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573577.591468:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616362624:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573577.591486:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573577.591499:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573577/real 1788573577] req@ffff9ada4557ca80 x1875449616362624/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573602 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573577.591513:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616362624/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573602 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573577.591528:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573577.591532:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573577.591537:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616362624:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573577.783014:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573578.103018:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573578.806943:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573579.127055:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573579.831006:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573580.150992:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573580.854925:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573581.175328:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573581.879039:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573582.199065:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573582.710948:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573582.710959:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573582.710964:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573582.710973:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5495 00000100:00080000:2.0:1788573582.710984:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573582.710992:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573582.711049:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616362752/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573582.711077:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573582.711086:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573582.711089:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573582.711092:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5495 00000100:00080000:2.0:1788573582.711096:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573582.711100:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573582.711110:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573582.711125:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616362880/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573582.711136:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573582.711139:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573582.711141:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573582.711145:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5495 00000100:00080000:2.0:1788573582.711148:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573582.711152:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573582.711157:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573582.711171:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616362752:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788573582.711171:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616363008/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573582.711314:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616362880:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573582.711378:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616363008:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573582.902962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573583.223043:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573583.927004:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573584.247035:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573584.950989:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573585.271098:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1788573585.783134:0:2371:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:0.0:1788573585.783174:0:2371:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1788573585.783187:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:0.0:1788573585.783229:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:0.0:1788573585.783239:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:1.0:1788573585.783287:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573582/real 1788573585] req@ffff9ada60624380 x1875449616363008/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573607 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573585.783317:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616363008/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573607 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573585.783331:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573585.783337:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573585.783342:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616363008:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573585.783350:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573582/real 1788573585] req@ffff9ada60626a00 x1875449616362752/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573607 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573585.783371:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616362752/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573607 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573585.783379:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573585.783383:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573585.783387:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616362752:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573585.783394:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573582/real 1788573585] req@ffff9ada60625500 x1875449616362880/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573607 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573585.783407:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616362880/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573607 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573585.783415:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573585.783419:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573585.783422:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616362880:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573585.974972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573586.295012:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573586.998971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573587.319017:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573587.831025:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573587.831034:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573587.831039:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573587.831048:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5500 00000100:00080000:0.0:1788573587.831058:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573587.831065:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573587.831100:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616363136/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573587.831129:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573587.831133:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573587.831135:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573587.831139:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5500 00000100:00080000:0.0:1788573587.831142:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573587.831147:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573587.831156:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573587.831167:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616363264/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573587.831179:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573587.831182:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573587.831184:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573587.831189:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5500 00000100:00080000:0.0:1788573587.831194:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573587.831198:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573587.831204:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573587.831206:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616363136:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788573587.831217:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616363392/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573587.831273:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573587.831332:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573587/real 1788573587] req@ffff9ada4557f800 x1875449616363136/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573612 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573587.831352:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616363136/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573612 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573587.831363:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573587.831367:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573587.831371:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616363136:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573587.831378:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616363264:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573587.831404:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573587.831418:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573587/real 1788573587] req@ffff9ada4557d880 x1875449616363264/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573612 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573587.831431:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616363264/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573612 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573587.831440:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573587.831444:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573587.831448:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616363264:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573587.831474:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616363392:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573587.831495:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573587.831508:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573587/real 1788573587] req@ffff9ada4557fb80 x1875449616363392/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573612 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573587.831536:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616363392/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573612 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573587.831544:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573587.831548:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573587.831551:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616363392:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573588.022953:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573588.343043:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573589.046963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573589.367041:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573590.070956:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573590.390999:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573591.094922:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573591.415006:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573592.118967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573592.439052:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573592.950980:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573592.950995:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573592.950999:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573592.951007:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5505 00000100:00080000:2.0:1788573592.951015:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573592.951021:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573592.951053:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616363520/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573592.951073:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573592.951078:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573592.951079:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573592.951082:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5505 00000100:00080000:2.0:1788573592.951084:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573592.951086:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573592.951094:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573592.951102:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616363648/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573592.951111:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573592.951114:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573592.951115:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1788573592.951121:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616363520:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788573592.951121:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5505 00000100:00080000:2.0:1788573592.951125:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573592.951131:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573592.951134:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573592.951140:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616363776/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573592.951193:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573592.951228:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573592/real 1788573592] req@ffff9ada60624a80 x1875449616363520/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573617 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573592.951244:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616363520/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573617 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573592.951252:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573592.951262:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573592.951265:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616363520:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573592.951285:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616363648:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573592.951310:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573592.951324:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573592/real 1788573592] req@ffff9ada60625880 x1875449616363648/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573617 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573592.951339:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616363648/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573617 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573592.951346:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573592.951349:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573592.951351:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616363648:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573592.951356:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616363776:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573592.951372:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573592.951380:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573592/real 1788573592] req@ffff9ada60625c00 x1875449616363776/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573617 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573592.951392:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616363776/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573617 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573592.951398:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573592.951399:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573592.951401:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616363776:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573593.142933:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573593.462994:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573594.166965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573594.487052:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573595.190974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573595.511014:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573596.214960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573596.535025:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573597.238958:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573597.559002:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573598.070970:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573598.070980:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573598.070983:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573598.070992:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5510 00000100:00080000:0.0:1788573598.071000:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573598.071006:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573598.071038:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616363904/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573598.071062:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573598.071066:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573598.071068:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573598.071072:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5510 00000100:00080000:0.0:1788573598.071076:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573598.071080:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573598.071089:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573598.071105:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616364032/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573598.071121:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616363904:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788573598.071128:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573598.071131:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573598.071134:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573598.071141:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5510 00000100:00080000:0.0:1788573598.071146:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573598.071150:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573598.071156:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573598.071167:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616364160/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573598.071214:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573598.071260:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573598/real 1788573598] req@ffff9ada4557dc00 x1875449616363904/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573623 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573598.071281:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616363904/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573623 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573598.071293:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573598.071298:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573598.071302:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616363904:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573598.071337:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616364032:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573598.071364:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573598.071378:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573598/real 1788573598] req@ffff9ada4557e300 x1875449616364032/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573623 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573598.071391:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616364032/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573623 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573598.071409:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573598.071413:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573598.071417:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616364032:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573598.071424:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616364160:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573598.071444:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573598.071456:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573598/real 1788573598] req@ffff9ada4557ed80 x1875449616364160/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573623 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573598.071464:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616364160/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573623 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573598.071470:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573598.071473:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573598.071475:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616364160:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573598.262966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573598.583074:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573599.286989:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573599.607039:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573600.310945:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573600.631051:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573601.334958:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573601.655046:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573602.358956:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573602.679001:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573603.190994:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573603.191005:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573603.191009:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573603.191019:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5515 00000100:00080000:3.0:1788573603.191029:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573603.191036:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573603.191077:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2a00 x1875449616364288/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573603.191105:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573603.191109:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573603.191111:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573603.191115:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5515 00000100:00080000:3.0:1788573603.191119:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573603.191124:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573603.191134:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573603.191147:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616364288:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573603.191153:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0e00 x1875449616364416/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573603.191167:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573603.191172:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573603.191174:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573603.191178:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5515 00000100:00080000:3.0:1788573603.191183:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573603.191186:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573603.191190:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573603.191201:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1500 x1875449616364544/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573603.191209:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573603.191253:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573603/real 1788573603] req@ffff9ada604e2a00 x1875449616364288/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573628 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573603.191268:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2a00 x1875449616364288/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573628 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573603.191276:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573603.191279:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573603.191282:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616364288:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573603.191301:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616364416:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573603.191316:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573603.191325:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573603/real 1788573603] req@ffff9ada604e0e00 x1875449616364416/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573628 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573603.191334:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0e00 x1875449616364416/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573628 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573603.191339:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573603.191341:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573603.191344:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616364416:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573603.191347:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616364544:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573603.191358:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573603.191364:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573603/real 1788573603] req@ffff9ada604e1500 x1875449616364544/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573628 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573603.191371:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1500 x1875449616364544/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573628 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573603.191376:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573603.191378:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573603.191380:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616364544:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573603.382970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573603.703049:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573604.406976:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573604.727056:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573605.430964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573605.751027:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573606.455023:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573606.775074:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573607.478970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573607.798990:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573608.310953:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573608.310964:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573608.310968:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573608.310978:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5520 00000100:00080000:3.0:1788573608.310989:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573608.310998:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573608.311039:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2680 x1875449616364672/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573608.311069:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573608.311072:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573608.311077:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573608.311080:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5520 00000100:00080000:3.0:1788573608.311084:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573608.311087:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573608.311095:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573608.311106:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1f80 x1875449616364800/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573608.311114:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573608.311117:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573608.311118:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1788573608.311123:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616364672:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788573608.311123:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5520 00000100:00080000:3.0:1788573608.311129:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573608.311133:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573608.311137:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573608.311151:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3100 x1875449616364928/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573608.311185:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573608.311216:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573608/real 1788573608] req@ffff9ada604e2680 x1875449616364672/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573633 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573608.311230:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2680 x1875449616364672/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573633 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573608.311238:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573608.311241:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573608.311244:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616364672:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573608.311263:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616364800:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573608.311277:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573608.311285:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573608/real 1788573608] req@ffff9ada604e1f80 x1875449616364800/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573633 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573608.311293:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1f80 x1875449616364800/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573633 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573608.311299:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573608.311302:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573608.311304:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616364800:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573608.311307:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616364928:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573608.311322:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573608.311333:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573608/real 1788573608] req@ffff9ada604e3100 x1875449616364928/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573633 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573608.311346:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3100 x1875449616364928/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573633 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573608.311362:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573608.311366:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573608.311370:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616364928:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573608.502935:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573608.823045:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573609.526970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573609.847079:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573610.550961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573610.871038:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573611.574996:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573611.895038:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573612.598957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573612.919026:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573613.430950:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573613.430958:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573613.430971:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573613.430982:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5525 00000100:00080000:3.0:1788573613.430992:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573613.431001:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573613.431038:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3b80 x1875449616365056/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573613.431063:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573613.431065:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573613.431071:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573613.431074:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5525 00000100:00080000:3.0:1788573613.431077:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573613.431080:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573613.431087:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573613.431096:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1c00 x1875449616365184/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573613.431104:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573613.431107:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573613.431110:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1788573613.431111:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616365056:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788573613.431114:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5525 00000100:00080000:3.0:1788573613.431119:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573613.431123:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573613.431126:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573613.431133:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0000 x1875449616365312/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573613.431173:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573613.431202:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573613/real 1788573613] req@ffff9ada604e3b80 x1875449616365056/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573638 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573613.431215:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3b80 x1875449616365056/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573638 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573613.431223:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573613.431226:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573613.431229:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616365056:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573613.431247:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616365184:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573613.431261:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573613.431269:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573613/real 1788573613] req@ffff9ada604e1c00 x1875449616365184/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573638 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573613.431277:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1c00 x1875449616365184/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573638 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573613.431282:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573613.431285:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573613.431287:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616365184:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573613.431291:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616365312:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573613.431301:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573613.431306:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573613/real 1788573613] req@ffff9ada604e0000 x1875449616365312/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573638 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573613.431313:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0000 x1875449616365312/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573638 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573613.431318:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573613.431320:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573613.431322:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616365312:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573613.622992:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573613.943030:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573614.646988:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573614.967002:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573615.670967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573615.991013:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573616.694933:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573617.014980:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573617.718957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573618.039055:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573618.551001:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573618.551009:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573618.551013:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573618.551022:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5531 00000100:00080000:2.0:1788573618.551032:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573618.551038:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573618.551070:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616365440/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573618.551092:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573618.551095:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573618.551096:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573618.551098:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5531 00000100:00080000:2.0:1788573618.551101:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573618.551110:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573618.551118:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573618.551128:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616365568/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573618.551136:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573618.551138:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573618.551139:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573618.551144:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5531 00000100:00100000:1.0:1788573618.551145:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616365440:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788573618.551146:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573618.551153:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573618.551157:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573618.551165:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616365696/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573618.551215:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573618.551245:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573618/real 1788573618] req@ffff9ada60625f80 x1875449616365440/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573643 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573618.551259:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616365440/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573643 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573618.551267:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573618.551285:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573618.551289:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616365440:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573618.551309:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616365568:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573618.551327:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573618.551343:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573618/real 1788573618] req@ffff9ada60624e00 x1875449616365568/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573643 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573618.551352:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616365568/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573643 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573618.551357:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573618.551359:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573618.551362:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616365568:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573618.551365:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616365696:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573618.551377:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573618.551383:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573618/real 1788573618] req@ffff9ada60627480 x1875449616365696/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573643 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573618.551391:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616365696/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573643 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573618.551396:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573618.551397:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573618.551399:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616365696:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573618.742980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573619.063061:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573619.766947:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573620.087087:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573620.790989:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573621.111055:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573621.814994:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573622.135072:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573622.838991:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573623.159032:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573623.670928:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573623.670939:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573623.670944:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573623.670955:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5536 00000100:00080000:2.0:1788573623.670967:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573623.670975:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573623.671019:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616365824/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573623.671051:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573623.671054:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573623.671056:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573623.671062:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5536 00000100:00080000:2.0:1788573623.671065:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573623.671068:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573623.671078:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573623.671088:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616365952/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573623.671095:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573623.671099:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573623.671100:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573623.671102:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5536 00000100:00080000:2.0:1788573623.671104:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573623.671106:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573623.671110:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573623.671113:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616365824:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788573623.671121:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616366080/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573623.671227:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573623.671282:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573623/real 1788573623] req@ffff9ada60625180 x1875449616365824/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573648 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573623.671314:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616365824/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573648 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573623.671327:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573623.671332:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573623.671337:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616365824:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573623.671346:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616365952:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573623.671372:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573623.671387:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573623/real 1788573623] req@ffff9ada60626d80 x1875449616365952/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573648 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573623.671401:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616365952/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573648 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573623.671411:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573623.671414:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573623.671419:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616365952:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573623.671447:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616366080:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573623.671469:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573623.671481:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573623/real 1788573623] req@ffff9ada60627b80 x1875449616366080/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573648 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573623.671495:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616366080/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573648 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573623.671511:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573623.671515:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573623.671519:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616366080:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573623.862965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573624.183032:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573624.886984:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573625.207266:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573625.910980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573626.231106:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573626.934996:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573627.255038:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573627.959012:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573628.279048:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573628.791003:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573628.791016:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573628.791020:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573628.791032:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5541 00000100:00080000:3.0:1788573628.791045:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573628.791052:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573628.791105:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0a80 x1875449616366208/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573628.791159:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573628.791163:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573628.791165:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573628.791168:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5541 00000100:00080000:3.0:1788573628.791171:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573628.791175:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573628.791186:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573628.791197:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2d80 x1875449616366336/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573628.791205:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573628.791209:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573628.791210:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573628.791214:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5541 00000100:00080000:3.0:1788573628.791216:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573628.791218:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573628.791222:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573628.791230:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616366208:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573628.791231:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2300 x1875449616366464/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573628.791331:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573628.791380:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573628/real 1788573628] req@ffff9ada604e0a80 x1875449616366208/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573653 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573628.791404:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0a80 x1875449616366208/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573653 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573628.791418:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573628.791423:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573628.791428:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616366208:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573628.791438:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616366336:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573628.791465:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573628.791493:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573628/real 1788573628] req@ffff9ada604e2d80 x1875449616366336/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573653 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573628.791508:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2d80 x1875449616366336/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573653 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573628.791518:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573628.791523:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573628.791527:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616366336:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573628.791558:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616366464:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573628.791593:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573628.791608:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573628/real 1788573628] req@ffff9ada604e2300 x1875449616366464/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573653 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573628.791624:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2300 x1875449616366464/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573653 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573628.791633:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573628.791637:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573628.791641:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616366464:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573628.982966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573629.303053:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573630.007002:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573630.327041:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573631.030945:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573631.351027:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573632.054943:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573632.375013:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573633.078950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573633.399048:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788573633.910991:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573633.911002:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788573633.911007:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573633.911017:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5546 00000100:00080000:1.0:1788573633.911028:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573633.911036:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788573633.911107:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d500 x1875449616366592/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573633.911136:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573633.911140:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788573633.911142:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573633.911147:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5546 00000100:00080000:1.0:1788573633.911152:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573633.911157:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573633.911167:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573633.911180:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616366592:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573633.911184:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d180 x1875449616366720/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573633.911199:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573633.911203:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788573633.911205:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573633.911209:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5546 00000100:00080000:1.0:1788573633.911214:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573633.911218:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573633.911224:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573633.911238:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ca80 x1875449616366848/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573633.911250:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573633.911289:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573633/real 1788573633] req@ffff9ada4566d500 x1875449616366592/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573658 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573633.911311:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d500 x1875449616366592/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573658 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573633.911323:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573633.911336:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573633.911345:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616366592:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573633.911370:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616366720:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573633.911396:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573633.911411:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573633/real 1788573633] req@ffff9ada4566d180 x1875449616366720/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573658 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573633.911424:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d180 x1875449616366720/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573658 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573633.911434:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573633.911437:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573633.911441:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616366720:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573633.911448:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616366848:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573633.911470:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573633.911482:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573633/real 1788573633] req@ffff9ada4566ca80 x1875449616366848/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573658 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573633.911495:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ca80 x1875449616366848/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573658 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573633.911504:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573633.911508:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573633.911512:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616366848:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573634.102963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573634.422991:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573635.126941:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573635.447055:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573636.150959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573636.471073:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573637.174966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573637.495096:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573638.199015:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573638.519129:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573639.031018:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573639.031027:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573639.031030:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573639.031039:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5551 00000100:00080000:2.0:1788573639.031047:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573639.031053:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573639.031086:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627800 x1875449616366976/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573639.031112:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573639.031116:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573639.031118:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573639.031122:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5551 00000100:00080000:2.0:1788573639.031126:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573639.031131:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573639.031141:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573639.031156:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626300 x1875449616367104/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573639.031173:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1788573639.031173:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616366976:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788573639.031186:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573639.031192:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573639.031196:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5551 00000100:00080000:2.0:1788573639.031200:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573639.031203:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573639.031210:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573639.031222:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616367232/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573639.031268:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573639.031309:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573639/real 1788573639] req@ffff9ada60627800 x1875449616366976/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573664 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573639.031326:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627800 x1875449616366976/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573664 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573639.031335:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573639.031339:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573639.031343:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616366976:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573639.031369:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616367104:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573639.031385:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573639.031393:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573639/real 1788573639] req@ffff9ada60626300 x1875449616367104/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573664 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573639.031402:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626300 x1875449616367104/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573664 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573639.031408:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573639.031410:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573639.031412:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616367104:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573639.031416:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616367232:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573639.031427:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573639.031433:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573639/real 1788573639] req@ffff9ada60627b80 x1875449616367232/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573664 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573639.031441:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616367232/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573664 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573639.031446:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573639.031449:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573639.031451:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616367232:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573639.222946:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573639.543078:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573640.247003:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573640.567089:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573641.270924:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573641.591051:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573642.294980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573642.615024:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573643.318989:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573643.639023:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573644.151031:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573644.151039:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573644.151043:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573644.151051:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5556 00000100:00080000:0.0:1788573644.151060:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573644.151066:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573644.151099:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616367360/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573644.151123:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573644.151126:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573644.151128:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573644.151131:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5556 00000100:00080000:0.0:1788573644.151133:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573644.151135:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573644.151142:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573644.151151:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616367488/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573644.151158:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573644.151161:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573644.151163:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573644.151165:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5556 00000100:00080000:0.0:1788573644.151168:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1788573644.151171:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616367360:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788573644.151174:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573644.151181:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573644.151188:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d180 x1875449616367616/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573644.151229:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573644.151256:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573644/real 1788573644] req@ffff9ada4557ea00 x1875449616367360/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573669 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573644.151271:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616367360/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573669 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573644.151279:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573644.151282:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573644.151285:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616367360:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573644.151290:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616367488:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573644.151304:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573644.151312:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573644/real 1788573644] req@ffff9ada4557e680 x1875449616367488/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573669 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573644.151320:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616367488/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573669 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573644.151326:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573644.151328:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573644.151330:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616367488:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573644.151347:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616367616:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573644.151357:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573644.151363:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573644/real 1788573644] req@ffff9ada4557d180 x1875449616367616/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573669 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573644.151376:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d180 x1875449616367616/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573669 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573644.151381:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573644.151383:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573644.151385:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616367616:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573644.342975:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573644.663011:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573645.366983:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573645.687066:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573646.390951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573646.711022:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573647.415055:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573647.735034:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573648.438950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573648.759000:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573649.270912:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573649.270921:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573649.270925:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573649.270933:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5561 00000100:00080000:0.0:1788573649.270943:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573649.270949:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573649.270981:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557df80 x1875449616367744/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573649.271007:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573649.271010:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573649.271015:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573649.271018:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5561 00000100:00080000:0.0:1788573649.271020:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573649.271023:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573649.271030:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573649.271039:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d500 x1875449616367872/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573649.271047:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573649.271049:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573649.271051:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573649.271054:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5561 00000100:00080000:0.0:1788573649.271056:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573649.271061:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573649.271063:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616367744:192.168.202.113@tcp:250:kworker.0 00000008:00100000:0.0:1788573649.271066:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573649.271077:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d180 x1875449616368000/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788573649.271148:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616367872:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573649.271176:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616368000:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573649.462942:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573649.783041:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573650.486937:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573650.807012:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573651.510933:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573651.831009:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1788573652.343158:0:2370:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:2.0:1788573652.343208:0:2370:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1788573652.343225:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788573652.343275:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788573652.343287:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:3.0:1788573652.343334:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573649/real 1788573652] req@ffff9ada4557d180 x1875449616368000/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573674 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573652.343377:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d180 x1875449616368000/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573674 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573652.343390:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573652.343396:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573652.343401:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616368000:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573652.343413:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573649/real 1788573652] req@ffff9ada4557df80 x1875449616367744/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573674 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573652.343427:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557df80 x1875449616367744/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573674 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573652.343435:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573652.343438:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573652.343442:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616367744:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573652.343448:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573649/real 1788573652] req@ffff9ada4557d500 x1875449616367872/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573674 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573652.343459:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d500 x1875449616367872/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573674 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573652.343467:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573652.343470:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573652.343473:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616367872:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573652.534960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573652.855052:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573653.558957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573653.878993:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573654.390946:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573654.390954:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573654.390958:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573654.390967:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5566 00000100:00080000:3.0:1788573654.390977:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573654.390984:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573654.391014:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1180 x1875449616368128/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573654.391039:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573654.391041:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573654.391043:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573654.391046:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5566 00000100:00080000:3.0:1788573654.391048:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573654.391054:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573654.391061:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573654.391075:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1880 x1875449616368256/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573654.391083:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573654.391085:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573654.391086:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573654.391089:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5566 00000100:00080000:3.0:1788573654.391091:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573654.391093:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573654.391097:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573654.391101:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616368128:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573654.391107:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3800 x1875449616368384/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573654.391173:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573654.391208:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573654/real 1788573654] req@ffff9ada604e1180 x1875449616368128/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573679 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573654.391228:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1180 x1875449616368128/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573679 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573654.391240:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573654.391245:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573654.391249:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616368128:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573654.391257:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616368256:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573654.391280:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573654.391310:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573654/real 1788573654] req@ffff9ada604e1880 x1875449616368256/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573679 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573654.391327:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1880 x1875449616368256/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573679 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573654.391336:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573654.391340:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573654.391343:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616368256:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573654.391370:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616368384:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573654.391392:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573654.391405:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573654/real 1788573654] req@ffff9ada604e3800 x1875449616368384/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573679 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573654.391418:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3800 x1875449616368384/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573679 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573654.391432:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573654.391436:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573654.391440:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616368384:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573654.582960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573654.903075:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573655.606967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573655.927007:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573656.630951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573656.951061:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573657.654968:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573657.975027:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573658.678915:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573658.999009:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573659.511023:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573659.511034:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573659.511038:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573659.511050:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5572 00000100:00080000:0.0:1788573659.511063:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573659.511071:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573659.511118:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616368512/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573659.511171:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573659.511175:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573659.511176:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573659.511180:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5572 00000100:00080000:0.0:1788573659.511183:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573659.511186:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573659.511199:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573659.511232:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616368640/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573659.511255:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573659.511260:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573659.511262:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573659.511266:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5572 00000100:00080000:0.0:1788573659.511270:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573659.511276:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788573659.511278:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616368512:192.168.202.113@tcp:250:kworker.0 00000008:00100000:0.0:1788573659.511282:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573659.511298:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616368768/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573659.511378:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573659.511418:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573659/real 1788573659] req@ffff9ada4557e680 x1875449616368512/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573684 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573659.511440:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616368512/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573684 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573659.511452:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573659.511457:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573659.511461:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616368512:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573659.511487:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616368640:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573659.511511:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573659.511525:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573659/real 1788573659] req@ffff9ada4557ea00 x1875449616368640/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573684 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573659.511538:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616368640/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573684 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573659.511548:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573659.511553:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573659.511557:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616368640:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573659.511564:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616368768:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573659.511617:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573659.511631:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573659/real 1788573659] req@ffff9ada4557ed80 x1875449616368768/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573684 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573659.511645:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616368768/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573684 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573659.511654:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573659.511657:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573659.511661:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616368768:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573659.702929:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573660.023197:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573660.726906:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573661.047080:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573661.750969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573662.071032:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573662.774964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573663.095027:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573663.798950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573664.119011:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573664.630961:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573664.630971:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573664.630976:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573664.630987:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5577 00000100:00080000:3.0:1788573664.630998:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573664.631006:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573664.631044:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0380 x1875449616368896/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573664.631075:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573664.631079:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573664.631081:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573664.631085:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5577 00000100:00080000:3.0:1788573664.631088:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573664.631091:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573664.631101:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573664.631111:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0700 x1875449616369024/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573664.631120:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1788573664.631122:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616368896:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788573664.631123:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573664.631127:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573664.631132:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5577 00000100:00080000:3.0:1788573664.631134:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573664.631136:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573664.631140:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573664.631149:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3480 x1875449616369152/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573664.631180:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573664.631211:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573664/real 1788573664] req@ffff9ada604e0380 x1875449616368896/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573689 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573664.631224:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0380 x1875449616368896/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573689 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573664.631232:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573664.631234:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573664.631238:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616368896:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573664.631267:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616369024:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573664.631283:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573664.631291:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573664/real 1788573664] req@ffff9ada604e0700 x1875449616369024/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573689 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573664.631299:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0700 x1875449616369024/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573689 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573664.631305:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573664.631307:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573664.631309:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616369024:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573664.631312:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616369152:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573664.631322:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573664.631329:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573664/real 1788573664] req@ffff9ada604e3480 x1875449616369152/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573689 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573664.631341:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3480 x1875449616369152/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573689 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573664.631350:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573664.631354:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573664.631357:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616369152:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573664.822946:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573665.143049:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573665.846956:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573666.167032:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573666.870960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573667.191016:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573667.894941:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573668.215074:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573668.918951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573669.239096:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573669.750989:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573669.750996:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573669.751000:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573669.751008:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5582 00000100:00080000:0.0:1788573669.751017:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573669.751023:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573669.751063:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616369280/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573669.751084:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573669.751088:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573669.751090:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573669.751094:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5582 00000100:00080000:0.0:1788573669.751098:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573669.751102:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573669.751112:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573669.751126:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616369408/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573669.751136:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1788573669.751136:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616369280:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788573669.751140:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573669.751141:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573669.751145:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5582 00000100:00080000:0.0:1788573669.751148:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573669.751152:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573669.751156:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573669.751165:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616369536/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573669.751194:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573669.751228:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573669/real 1788573669] req@ffff9ada4557e300 x1875449616369280/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573694 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573669.751243:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616369280/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573694 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573669.751250:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573669.751253:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573669.751255:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616369280:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573669.751278:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616369408:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573669.751302:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573669.751316:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573669/real 1788573669] req@ffff9ada4557dc00 x1875449616369408/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573694 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573669.751344:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616369408/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573694 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573669.751350:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573669.751352:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573669.751355:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616369408:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573669.751359:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616369536:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573669.751376:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573669.751384:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573669/real 1788573669] req@ffff9ada4557fb80 x1875449616369536/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573694 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573669.751392:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616369536/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573694 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573669.751402:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573669.751404:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573669.751406:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616369536:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573669.942953:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573670.263020:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573670.966947:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573671.287012:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573671.990965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573672.311073:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573673.014948:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573673.334992:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573674.038949:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573674.359061:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788573674.870948:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573674.870957:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788573674.870961:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573674.870969:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5587 00000100:00080000:1.0:1788573674.870978:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573674.870985:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788573674.871017:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f480 x1875449616369664/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573674.871042:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573674.871045:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788573674.871047:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573674.871050:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5587 00000100:00080000:1.0:1788573674.871052:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573674.871054:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573674.871063:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573674.871072:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e300 x1875449616369792/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573674.871081:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573674.871085:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1788573674.871085:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616369664:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788573674.871088:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573674.871095:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5587 00000100:00080000:1.0:1788573674.871098:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573674.871100:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573674.871112:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573674.871121:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ea00 x1875449616369920/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573674.871173:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573674.871233:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573674/real 1788573674] req@ffff9ada4566f480 x1875449616369664/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573699 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573674.871249:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f480 x1875449616369664/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573699 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573674.871257:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573674.871260:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573674.871263:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616369664:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573674.871294:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616369792:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573674.871312:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573674.871321:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573674/real 1788573674] req@ffff9ada4566e300 x1875449616369792/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573699 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573674.871329:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e300 x1875449616369792/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573699 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573674.871335:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573674.871337:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573674.871339:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616369792:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573674.871343:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616369920:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573674.871353:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573674.871359:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573674/real 1788573674] req@ffff9ada4566ea00 x1875449616369920/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573699 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573674.871367:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ea00 x1875449616369920/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573699 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573674.871372:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573674.871373:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573674.871375:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616369920:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573675.062954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573675.382976:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573676.086951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573676.407037:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573677.110964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573677.430960:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573678.134967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573678.455011:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573679.158948:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573679.479006:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788573679.990935:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573679.990944:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788573679.990947:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573679.990955:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5592 00000100:00080000:1.0:1788573679.990964:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573679.990970:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788573679.991006:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ed80 x1875449616370048/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573679.991035:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573679.991040:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788573679.991042:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573679.991046:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5592 00000100:00080000:1.0:1788573679.991050:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573679.991053:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573679.991061:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573679.991085:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f800 x1875449616370176/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573679.991094:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573679.991096:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788573679.991098:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573679.991101:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5592 00000100:00080000:1.0:1788573679.991104:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573679.991106:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573679.991110:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573679.991121:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566dc00 x1875449616370304/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788573679.991121:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616370048:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788573679.991193:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573679.991226:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573679/real 1788573679] req@ffff9ada4566ed80 x1875449616370048/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573704 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573679.991247:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ed80 x1875449616370048/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573704 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573679.991260:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573679.991264:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573679.991268:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616370048:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573679.991279:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616370176:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573679.991302:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573679.991315:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573679/real 1788573679] req@ffff9ada4566f800 x1875449616370176/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573704 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573679.991330:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f800 x1875449616370176/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573704 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573679.991339:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573679.991342:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573679.991346:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616370176:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573679.991370:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616370304:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573679.991390:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573679.991401:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573679/real 1788573679] req@ffff9ada4566dc00 x1875449616370304/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573704 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573679.991414:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566dc00 x1875449616370304/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573704 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573679.991423:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573679.991426:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573679.991430:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616370304:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573680.182941:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573680.503025:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573681.206994:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573681.527056:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573682.230970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573682.551027:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573683.254963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573683.575022:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573684.279019:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573684.599075:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573685.111006:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573685.111014:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573685.111019:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573685.111028:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5597 00000100:00080000:2.0:1788573685.111038:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573685.111046:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573685.111091:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616370432/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573685.111115:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573685.111118:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573685.111119:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573685.111122:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5597 00000100:00080000:2.0:1788573685.111125:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573685.111127:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573685.111135:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573685.111146:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616370560/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573685.111154:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573685.111157:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573685.111158:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573685.111161:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5597 00000100:00080000:2.0:1788573685.111164:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1788573685.111167:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616370432:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788573685.111169:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573685.111177:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573685.111185:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616370688/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573685.111225:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573685.111253:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573685/real 1788573685] req@ffff9ada60626d80 x1875449616370432/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573710 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573685.111267:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616370432/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573710 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573685.111276:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573685.111279:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573685.111283:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616370432:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573685.111290:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616370560:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573685.111314:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573685.111327:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573685/real 1788573685] req@ffff9ada60625180 x1875449616370560/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573710 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573685.111340:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616370560/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573710 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573685.111348:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573685.111352:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573685.111355:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616370560:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573685.111379:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616370688:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573685.111398:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573685.111421:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573685/real 1788573685] req@ffff9ada60627480 x1875449616370688/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573710 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573685.111433:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616370688/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573710 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573685.111441:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573685.111444:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573685.111447:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616370688:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573685.302998:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573685.623081:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573686.327013:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573686.647047:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573687.350999:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573687.671053:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573688.374955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573688.695317:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573689.398957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573689.719046:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573690.230927:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573690.230936:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573690.230939:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573690.230948:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5602 00000100:00080000:2.0:1788573690.230957:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573690.230963:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573690.230997:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616370816/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573690.231027:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573690.231032:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573690.231034:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573690.231038:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5602 00000100:00080000:2.0:1788573690.231042:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573690.231047:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573690.231058:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573690.231071:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616370944/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573690.231085:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573690.231087:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573690.231090:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1788573690.231093:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616370816:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788573690.231095:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5602 00000100:00080000:2.0:1788573690.231101:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573690.231105:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573690.231109:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573690.231116:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616371072/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573690.231165:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573690.231198:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573690/real 1788573690] req@ffff9ada60624e00 x1875449616370816/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573715 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573690.231217:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616370816/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573715 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573690.231228:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573690.231233:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573690.231236:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616370816:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573690.231261:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616370944:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573690.231283:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573690.231298:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573690/real 1788573690] req@ffff9ada60625f80 x1875449616370944/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573715 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573690.231313:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616370944/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573715 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573690.231322:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573690.231326:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573690.231329:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616370944:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573690.231336:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616371072:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573690.231357:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573690.231367:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573690/real 1788573690] req@ffff9ada60625c00 x1875449616371072/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573715 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573690.231376:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616371072/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573715 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573690.231381:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573690.231383:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573690.231385:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616371072:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573690.422950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573690.743004:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573691.446955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573691.767031:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573692.471002:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573692.790987:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573693.494967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573693.815008:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573694.518955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573694.838969:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573695.350959:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573695.350967:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573695.350970:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573695.350979:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5607 00000100:00080000:2.0:1788573695.350988:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573695.350994:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573695.351027:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616371200/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573695.351053:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573695.351056:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573695.351058:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573695.351061:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5607 00000100:00080000:2.0:1788573695.351063:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573695.351066:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573695.351073:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573695.351082:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616371328/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573695.351089:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573695.351092:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573695.351093:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573695.351095:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5607 00000100:00080000:2.0:1788573695.351097:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573695.351099:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573695.351102:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573695.351106:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616371200:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788573695.351112:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616371456/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573695.351161:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573695.351189:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573695/real 1788573695] req@ffff9ada60625880 x1875449616371200/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573720 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573695.351204:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616371200/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573720 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573695.351212:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573695.351215:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573695.351218:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616371200:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573695.351223:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616371328:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573695.351237:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573695.351245:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573695/real 1788573695] req@ffff9ada60624a80 x1875449616371328/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573720 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573695.351256:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616371328/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573720 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573695.351264:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573695.351267:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573695.351271:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616371328:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573695.351296:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616371456:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573695.351318:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573695.351329:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573695/real 1788573695] req@ffff9ada60625500 x1875449616371456/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573720 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573695.351337:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616371456/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573720 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573695.351347:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573695.351349:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573695.351351:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616371456:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573695.542959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573695.863260:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573696.566971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573696.887014:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573697.590924:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573697.911079:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573698.614963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573698.935026:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573699.638921:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573699.959040:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573700.471035:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573700.471043:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573700.471047:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573700.471056:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5613 00000100:00080000:3.0:1788573700.471065:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573700.471071:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573700.471124:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3480 x1875449616371584/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573700.471148:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573700.471150:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573700.471152:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573700.471155:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5613 00000100:00080000:3.0:1788573700.471157:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573700.471160:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573700.471167:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573700.471178:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0700 x1875449616371712/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573700.471191:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573700.471195:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:1.0:1788573700.471200:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616371584:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788573700.471200:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573700.471210:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5613 00000100:00080000:3.0:1788573700.471216:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573700.471220:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573700.471226:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573700.471239:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0380 x1875449616371840/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573700.471258:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573700.471290:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573700/real 1788573700] req@ffff9ada604e3480 x1875449616371584/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573725 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573700.471303:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3480 x1875449616371584/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573725 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573700.471311:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573700.471314:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573700.471317:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616371584:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573700.471335:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616371712:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573700.471350:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573700.471359:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573700/real 1788573700] req@ffff9ada604e0700 x1875449616371712/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573725 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573700.471367:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0700 x1875449616371712/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573725 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573700.471373:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573700.471375:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573700.471378:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616371712:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573700.471381:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616371840:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573700.471391:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573700.471397:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573700/real 1788573700] req@ffff9ada604e0380 x1875449616371840/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573725 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573700.471405:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0380 x1875449616371840/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573725 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573700.471410:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573700.471412:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573700.471414:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616371840:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573700.662947:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573700.983013:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573701.686954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573702.007100:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573702.710936:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573703.031092:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573703.734963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573704.055004:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573704.758947:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573705.078992:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573705.590918:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573705.590933:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573705.590937:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573705.590945:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5618 00000100:00080000:3.0:1788573705.590954:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573705.590961:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573705.590992:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3800 x1875449616371968/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573705.591013:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573705.591016:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573705.591018:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573705.591020:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5618 00000100:00080000:3.0:1788573705.591023:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573705.591025:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573705.591033:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573705.591041:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1880 x1875449616372096/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573705.591048:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573705.591050:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573705.591052:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573705.591054:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5618 00000100:00080000:3.0:1788573705.591056:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573705.591059:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573705.591062:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573705.591068:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616371968:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573705.591071:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1180 x1875449616372224/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573705.591125:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573705.591155:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573705/real 1788573705] req@ffff9ada604e3800 x1875449616371968/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573730 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573705.591168:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3800 x1875449616371968/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573730 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573705.591180:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573705.591183:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573705.591186:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616371968:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573705.591190:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616372096:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573705.591204:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573705.591212:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573705/real 1788573705] req@ffff9ada604e1880 x1875449616372096/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573730 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573705.591220:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1880 x1875449616372096/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573730 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573705.591225:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573705.591227:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573705.591229:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616372096:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573705.591248:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616372224:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573705.591259:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573705.591265:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573705/real 1788573705] req@ffff9ada604e1180 x1875449616372224/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573730 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573705.591272:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1180 x1875449616372224/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573730 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573705.591278:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573705.591279:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573705.591281:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616372224:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573705.782949:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573706.103055:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573706.806962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573707.126973:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573707.831010:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573708.151001:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573708.854963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573709.174998:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573709.878966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573710.199003:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573710.711015:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573710.711023:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573710.711027:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573710.711036:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5623 00000100:00080000:2.0:1788573710.711045:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573710.711052:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573710.711087:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616372352/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573710.711114:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573710.711118:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573710.711120:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573710.711123:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5623 00000100:00080000:2.0:1788573710.711127:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573710.711134:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573710.711141:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573710.711151:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616372480/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573710.711161:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573710.711165:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:1.0:1788573710.711166:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616372352:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788573710.711167:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573710.711183:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5623 00000100:00080000:2.0:1788573710.711186:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573710.711189:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573710.711193:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573710.711203:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616372608/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573710.711261:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573710.711326:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573710/real 1788573710] req@ffff9ada60626a00 x1875449616372352/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573735 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573710.711345:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616372352/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573735 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573710.711355:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573710.711359:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573710.711362:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616372352:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573710.711387:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616372480:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573710.711405:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573710.711414:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573710/real 1788573710] req@ffff9ada60624380 x1875449616372480/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573735 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573710.711422:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616372480/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573735 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573710.711428:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573710.711431:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573710.711433:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616372480:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573710.711437:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616372608:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573710.711449:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573710.711456:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573710/real 1788573710] req@ffff9ada60626680 x1875449616372608/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573735 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573710.711464:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616372608/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573735 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573710.711469:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573710.711471:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573710.711473:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616372608:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573710.902975:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573711.223055:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573711.926954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573712.247021:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573712.950947:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573713.271091:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573713.974987:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573714.295091:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573714.998929:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573715.319096:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573715.830990:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573715.831000:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573715.831004:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573715.831012:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5628 00000100:00080000:3.0:1788573715.831021:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573715.831026:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573715.831060:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2300 x1875449616372736/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573715.831087:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573715.831090:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573715.831092:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573715.831094:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5628 00000100:00080000:3.0:1788573715.831097:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573715.831099:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573715.831106:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573715.831114:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2d80 x1875449616372864/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573715.831122:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573715.831125:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573715.831126:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573715.831129:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5628 00000100:00080000:3.0:1788573715.831131:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573715.831134:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573715.831138:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573715.831143:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616372736:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573715.831147:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0a80 x1875449616372992/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573715.831249:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616372864:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573715.831280:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616372992:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573716.022970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573716.343039:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573717.046944:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573717.367141:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573718.070998:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573718.391036:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1788573718.903176:0:2372:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:0.0:1788573718.903213:0:2372:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1788573718.903226:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:0.0:1788573718.903270:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:0.0:1788573718.903283:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:1.0:1788573718.903353:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573715/real 1788573718] req@ffff9ada604e0a80 x1875449616372992/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573740 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573718.903381:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0a80 x1875449616372992/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573740 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573718.903394:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573718.903399:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573718.903404:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616372992:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573718.903412:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573715/real 1788573718] req@ffff9ada604e2300 x1875449616372736/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573740 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573718.903431:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2300 x1875449616372736/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573740 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573718.903439:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573718.903442:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573718.903446:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616372736:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573718.903452:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573715/real 1788573718] req@ffff9ada604e2d80 x1875449616372864/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573740 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573718.903462:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2d80 x1875449616372864/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573740 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573718.903470:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573718.903474:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573718.903477:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616372864:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573719.094979:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573719.415033:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573720.119047:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573720.439102:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573720.950973:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573720.950989:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573720.950995:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573720.951009:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5633 00000100:00080000:2.0:1788573720.951024:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573720.951034:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573720.951110:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624700 x1875449616373120/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573720.951145:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573720.951155:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573720.951158:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573720.951163:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5633 00000100:00080000:2.0:1788573720.951167:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573720.951172:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573720.951185:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573720.951203:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627100 x1875449616373248/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573720.951216:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573720.951220:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573720.951223:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573720.951229:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5633 00000100:00080000:2.0:1788573720.951234:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573720.951243:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573720.951249:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573720.951261:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616373120:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788573720.951271:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616373376/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573720.951363:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573720.951418:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573720/real 1788573720] req@ffff9ada60624700 x1875449616373120/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573745 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573720.951438:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624700 x1875449616373120/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573745 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573720.951449:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573720.951454:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573720.951458:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616373120:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573720.951467:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616373248:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573720.951492:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573720.951506:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573720/real 1788573720] req@ffff9ada60627100 x1875449616373248/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573745 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573720.951518:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627100 x1875449616373248/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573745 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573720.951526:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573720.951530:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573720.951534:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616373248:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573720.951565:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616373376:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573720.951589:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573720.951601:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573720/real 1788573720] req@ffff9ada60626680 x1875449616373376/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573745 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573720.951614:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616373376/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573745 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573720.951623:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573720.951626:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573720.951629:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616373376:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573721.142976:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573721.463083:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573722.166934:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573722.486991:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573723.191004:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573723.511110:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573724.214985:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573724.535092:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573725.238958:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573725.559095:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573726.071031:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573726.071042:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573726.071048:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573726.071060:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5638 00000100:00080000:2.0:1788573726.071074:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573726.071082:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573726.071127:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616373504/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573726.071161:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573726.071164:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573726.071165:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573726.071168:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5638 00000100:00080000:2.0:1788573726.071171:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573726.071173:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573726.071183:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573726.071192:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616373632/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573726.071207:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573726.071212:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573726.071214:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573726.071217:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5638 00000100:00080000:2.0:1788573726.071220:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573726.071223:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788573726.071228:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616373504:192.168.202.113@tcp:250:kworker.0 00000008:00100000:2.0:1788573726.071229:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573726.071243:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616373760/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573726.071328:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573726.071369:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573726/real 1788573726] req@ffff9ada60624380 x1875449616373504/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573751 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573726.071386:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616373504/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573751 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573726.071395:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573726.071398:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573726.071402:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616373504:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573726.071407:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616373632:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573726.071424:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573726.071433:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573726/real 1788573726] req@ffff9ada60626a00 x1875449616373632/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573751 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573726.071463:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616373632/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573751 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573726.071469:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573726.071472:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573726.071474:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616373632:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573726.071499:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616373760:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573726.071513:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573726.071520:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573726/real 1788573726] req@ffff9ada60625500 x1875449616373760/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573751 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573726.071529:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616373760/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573751 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573726.071554:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573726.071557:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573726.071559:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616373760:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573726.263013:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573726.583055:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573727.287040:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573727.607018:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573728.310963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573728.631024:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573729.334958:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573729.655043:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573730.358986:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573730.679107:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573731.191002:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573731.191011:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573731.191014:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573731.191023:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5643 00000100:00080000:3.0:1788573731.191032:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573731.191038:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573731.191072:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0000 x1875449616373888/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573731.191098:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573731.191101:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573731.191103:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573731.191105:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5643 00000100:00080000:3.0:1788573731.191108:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573731.191110:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573731.191118:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573731.191128:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1c00 x1875449616374016/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573731.191136:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573731.191139:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573731.191140:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573731.191143:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5643 00000100:00080000:3.0:1788573731.191145:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573731.191148:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573731.191152:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573731.191163:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3b80 x1875449616374144/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573731.191171:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616373888:192.168.202.113@tcp:250:kworker.0 00000800:00000100:1.0:1788573731.191263:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573731.191311:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573731/real 1788573731] req@ffff9ada604e0000 x1875449616373888/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573756 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573731.191329:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0000 x1875449616373888/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573756 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573731.191339:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573731.191343:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573731.191347:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616373888:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573731.191353:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616374016:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573731.191372:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573731.191386:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573731/real 1788573731] req@ffff9ada604e1c00 x1875449616374016/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573756 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573731.191400:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1c00 x1875449616374016/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573756 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573731.191409:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573731.191413:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573731.191418:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616374016:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573731.191449:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616374144:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573731.191473:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573731.191497:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573731/real 1788573731] req@ffff9ada604e3b80 x1875449616374144/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573756 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573731.191511:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3b80 x1875449616374144/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573756 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573731.191527:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573731.191531:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573731.191534:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616374144:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573731.382989:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573731.703048:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573732.407025:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573732.727067:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573733.431010:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573733.750997:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573734.454942:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573734.775087:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573735.479004:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573735.799033:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573736.310960:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573736.310969:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573736.310973:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573736.310982:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5648 00000100:00080000:3.0:1788573736.310991:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573736.310997:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573736.311031:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3100 x1875449616374272/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573736.311055:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573736.311057:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573736.311059:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573736.311061:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5648 00000100:00080000:3.0:1788573736.311063:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573736.311066:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573736.311073:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573736.311085:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1f80 x1875449616374400/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573736.311092:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573736.311095:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573736.311096:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573736.311098:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5648 00000100:00080000:3.0:1788573736.311102:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573736.311104:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573736.311107:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573736.311115:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2680 x1875449616374528/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573736.311125:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616374272:192.168.202.113@tcp:250:kworker.0 00000800:00000100:1.0:1788573736.311178:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573736.311208:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573736/real 1788573736] req@ffff9ada604e3100 x1875449616374272/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573761 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573736.311221:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3100 x1875449616374272/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573761 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573736.311229:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573736.311232:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573736.311235:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616374272:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573736.311240:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616374400:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573736.311254:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573736.311261:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573736/real 1788573736] req@ffff9ada604e1f80 x1875449616374400/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573761 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573736.311270:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1f80 x1875449616374400/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573761 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573736.311275:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573736.311277:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573736.311281:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616374400:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573736.311307:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616374528:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573736.311338:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573736.311355:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573736/real 1788573736] req@ffff9ada604e2680 x1875449616374528/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573761 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573736.311365:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2680 x1875449616374528/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573761 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573736.311370:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573736.311373:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573736.311375:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616374528:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573736.502934:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573736.823048:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573737.526956:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573737.847007:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573738.550921:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573738.871047:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573739.574997:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573739.895081:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573740.599013:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573740.919067:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788573741.431017:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573741.431025:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788573741.431029:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573741.431037:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5653 00000100:00080000:1.0:1788573741.431047:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573741.431053:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788573741.431088:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c000 x1875449616374656/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573741.431114:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573741.431117:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788573741.431119:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573741.431121:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5653 00000100:00080000:1.0:1788573741.431124:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573741.431127:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573741.431136:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573741.431152:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566df80 x1875449616374784/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573741.431162:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573741.431170:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1788573741.431172:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616374656:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788573741.431174:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573741.431187:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5653 00000100:00080000:1.0:1788573741.431190:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573741.431192:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573741.431195:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573741.431206:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566fb80 x1875449616374912/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573741.431266:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573741.431311:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573741/real 1788573741] req@ffff9ada4566c000 x1875449616374656/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573766 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573741.431335:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c000 x1875449616374656/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573766 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573741.431355:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573741.431359:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573741.431362:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616374656:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573741.431389:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616374784:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573741.431409:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573741.431419:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573741/real 1788573741] req@ffff9ada4566df80 x1875449616374784/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573766 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573741.431428:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566df80 x1875449616374784/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573766 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573741.431434:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573741.431437:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573741.431440:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616374784:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573741.431444:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616374912:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573741.431456:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573741.431462:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573741/real 1788573741] req@ffff9ada4566fb80 x1875449616374912/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573766 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573741.431469:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566fb80 x1875449616374912/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573766 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573741.431475:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573741.431477:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573741.431479:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616374912:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573741.623007:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573741.943039:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573742.647014:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573742.967060:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573743.670970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573743.991043:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573744.694975:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573745.015149:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573745.719015:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573746.039086:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788573746.550951:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573746.550961:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788573746.550965:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573746.550973:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5659 00000100:00080000:1.0:1788573746.550981:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573746.550987:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788573746.551020:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566fb80 x1875449616375040/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573746.551044:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573746.551047:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788573746.551052:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573746.551055:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5659 00000100:00080000:1.0:1788573746.551058:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573746.551060:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573746.551067:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573746.551076:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566df80 x1875449616375168/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573746.551083:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573746.551086:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788573746.551087:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573746.551093:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5659 00000100:00080000:1.0:1788573746.551095:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573746.551100:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573746.551100:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616375040:192.168.202.113@tcp:250:kworker.0 00000008:00100000:1.0:1788573746.551108:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573746.551118:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c000 x1875449616375296/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573746.551159:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573746.551195:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573746/real 1788573746] req@ffff9ada4566fb80 x1875449616375040/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573771 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573746.551209:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566fb80 x1875449616375040/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573771 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573746.551217:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573746.551220:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573746.551223:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616375040:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573746.551228:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616375168:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573746.551245:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573746.551253:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573746/real 1788573746] req@ffff9ada4566df80 x1875449616375168/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573771 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573746.551260:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566df80 x1875449616375168/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573771 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573746.551266:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573746.551267:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573746.551269:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616375168:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573746.551288:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616375296:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573746.551300:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573746.551307:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573746/real 1788573746] req@ffff9ada4566c000 x1875449616375296/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573771 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573746.551314:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c000 x1875449616375296/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573771 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573746.551319:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573746.551322:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573746.551324:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616375296:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573746.742925:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573747.063090:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573747.766952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573748.087025:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573748.790942:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573749.111044:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573749.814990:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573750.135045:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573750.838924:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573751.159012:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573751.671006:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573751.671014:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573751.671017:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573751.671027:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5664 00000100:00080000:2.0:1788573751.671037:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573751.671051:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573751.671087:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616375424/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573751.671119:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573751.671123:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573751.671126:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573751.671131:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5664 00000100:00080000:2.0:1788573751.671135:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573751.671140:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573751.671149:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573751.671161:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616375552/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573751.671171:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1788573751.671171:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616375424:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788573751.671178:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573751.671181:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573751.671184:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5664 00000100:00080000:2.0:1788573751.671187:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573751.671189:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573751.671193:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573751.671201:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616375680/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573751.671233:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573751.671267:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573751/real 1788573751] req@ffff9ada60624a80 x1875449616375424/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573776 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573751.671289:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616375424/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573776 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573751.671301:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573751.671305:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573751.671309:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616375424:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573751.671333:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616375552:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573751.671355:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573751.671367:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573751/real 1788573751] req@ffff9ada60625880 x1875449616375552/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573776 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573751.671380:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616375552/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573776 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573751.671388:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573751.671391:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573751.671395:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616375552:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573751.671400:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616375680:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573751.671417:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573751.671426:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573751/real 1788573751] req@ffff9ada60625c00 x1875449616375680/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573776 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573751.671439:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616375680/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573776 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573751.671452:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573751.671456:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573751.671459:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616375680:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573751.862942:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573752.183016:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573752.886961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573753.207047:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573753.910972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573754.231105:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573754.934951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573755.254973:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573755.958939:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573756.279053:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573756.790955:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573756.790963:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573756.790967:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573756.790974:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5669 00000100:00080000:0.0:1788573756.790984:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573756.790990:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573756.791022:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616375808/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573756.791043:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573756.791046:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573756.791047:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573756.791050:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5669 00000100:00080000:0.0:1788573756.791052:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573756.791054:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573756.791061:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573756.791069:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616375936/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573756.791076:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573756.791079:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573756.791080:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573756.791082:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5669 00000100:00080000:0.0:1788573756.791084:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573756.791087:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573756.791090:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573756.791096:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616376064/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788573756.791102:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616375808:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788573756.791167:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573756.791200:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573756/real 1788573756] req@ffff9ada4557d880 x1875449616375808/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573781 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573756.791219:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616375808/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573781 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573756.791230:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573756.791234:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573756.791239:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616375808:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573756.791245:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616375936:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573756.791264:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573756.791274:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573756/real 1788573756] req@ffff9ada4557f800 x1875449616375936/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573781 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573756.791299:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616375936/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573781 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573756.791308:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573756.791312:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573756.791316:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616375936:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573756.791340:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616376064:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573756.791357:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573756.791380:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573756/real 1788573756] req@ffff9ada4557ca80 x1875449616376064/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573781 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573756.791394:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616376064/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573781 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573756.791403:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573756.791406:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573756.791410:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616376064:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573756.983151:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573757.303045:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573758.007004:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573758.327039:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573759.031006:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573759.351030:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573760.054994:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573760.375291:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573761.078961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573761.399022:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573761.910967:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573761.910975:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573761.910979:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573761.910992:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5674 00000100:00080000:0.0:1788573761.911001:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573761.911007:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573761.911038:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616376192/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573761.911059:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573761.911062:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573761.911064:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573761.911066:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5674 00000100:00080000:0.0:1788573761.911068:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573761.911070:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573761.911078:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573761.911087:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616376320/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573761.911094:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573761.911097:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573761.911099:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573761.911103:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5674 00000100:00080000:0.0:1788573761.911107:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573761.911110:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573761.911113:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573761.911122:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616376192:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788573761.911125:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c000 x1875449616376448/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573761.911216:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573761.911258:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573761/real 1788573761] req@ffff9ada4557f480 x1875449616376192/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573786 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573761.911291:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616376192/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573786 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573761.911306:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573761.911310:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573761.911316:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616376192:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573761.911323:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616376320:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573761.911343:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573761.911353:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573761/real 1788573761] req@ffff9ada4557c700 x1875449616376320/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573786 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573761.911362:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616376320/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573786 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573761.911368:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573761.911370:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573761.911373:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616376320:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573761.911398:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616376448:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573761.911409:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573761.911416:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573761/real 1788573761] req@ffff9ada4557c000 x1875449616376448/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573786 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573761.911424:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c000 x1875449616376448/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573786 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573761.911429:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573761.911431:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573761.911433:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616376448:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573762.103000:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573762.423097:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573763.126962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573763.446995:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573764.150919:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573764.471025:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573765.174933:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573765.494983:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573766.198933:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573766.519029:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573767.031002:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573767.031016:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573767.031022:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573767.031035:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5679 00000100:00080000:0.0:1788573767.031050:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573767.031060:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573767.031113:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c380 x1875449616376576/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573767.031142:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573767.031145:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573767.031146:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573767.031150:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5679 00000100:00080000:0.0:1788573767.031153:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573767.031156:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573767.031177:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573767.031189:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ce00 x1875449616376704/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573767.031199:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573767.031202:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573767.031204:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573767.031208:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5679 00000100:00080000:0.0:1788573767.031213:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573767.031217:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573767.031223:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573767.031243:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c000 x1875449616376832/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788573767.031245:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616376576:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788573767.031344:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573767.031389:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573767/real 1788573767] req@ffff9ada4557c380 x1875449616376576/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573792 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573767.031412:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c380 x1875449616376576/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573792 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573767.031424:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573767.031430:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573767.031434:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616376576:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573767.031444:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616376704:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573767.031471:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573767.031486:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573767/real 1788573767] req@ffff9ada4557ce00 x1875449616376704/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573792 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573767.031500:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ce00 x1875449616376704/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573792 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573767.031509:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573767.031513:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573767.031517:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616376704:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573767.031542:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616376832:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573767.031565:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573767.031578:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573767/real 1788573767] req@ffff9ada4557c000 x1875449616376832/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573792 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573767.031591:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c000 x1875449616376832/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573792 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573767.031606:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573767.031610:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573767.031614:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616376832:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573767.222951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573767.543142:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573768.246920:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573768.567043:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573769.270926:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573769.591055:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573770.294926:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573770.615033:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573771.318940:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573771.639042:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573772.150991:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573772.151010:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573772.151015:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573772.151025:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5684 00000100:00080000:0.0:1788573772.151036:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573772.151044:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573772.151103:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616376960/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573772.151124:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573772.151126:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573772.151128:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573772.151131:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5684 00000100:00080000:0.0:1788573772.151135:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573772.151138:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573772.151147:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573772.151153:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616376960:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788573772.151158:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616377088/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573772.151167:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573772.151169:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573772.151171:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573772.151174:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5684 00000100:00080000:0.0:1788573772.151176:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573772.151178:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573772.151181:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573772.151189:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616377216/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573772.151210:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573772.151238:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573772/real 1788573772] req@ffff9ada4557c700 x1875449616376960/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573797 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573772.151253:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616376960/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573797 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573772.151261:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573772.151263:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573772.151266:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616376960:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573772.151285:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616377088:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573772.151300:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573772.151307:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573772/real 1788573772] req@ffff9ada4557f480 x1875449616377088/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573797 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573772.151315:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616377088/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573797 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573772.151321:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573772.151323:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573772.151325:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616377088:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573772.151330:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616377216:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573772.151340:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573772.151346:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573772/real 1788573772] req@ffff9ada4557ca80 x1875449616377216/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573797 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573772.151353:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616377216/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573797 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573772.151358:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573772.151360:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573772.151362:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616377216:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573772.342957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573772.663014:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573773.366928:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573773.687027:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573774.390930:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573774.711070:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573775.414945:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573775.734976:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573776.438949:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573776.759042:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573777.270989:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573777.270997:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573777.271000:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573777.271008:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5689 00000100:00080000:0.0:1788573777.271018:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573777.271024:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573777.271057:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616377344/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573777.271081:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573777.271083:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573777.271085:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573777.271088:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5689 00000100:00080000:0.0:1788573777.271090:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573777.271092:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573777.271099:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573777.271106:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616377472/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573777.271113:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573777.271116:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573777.271117:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573777.271120:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5689 00000100:00080000:0.0:1788573777.271122:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573777.271125:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573777.271127:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573777.271134:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616377600/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788573777.271144:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616377344:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788573777.271207:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573777.271237:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573777/real 1788573777] req@ffff9ada4557f800 x1875449616377344/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573802 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573777.271252:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616377344/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573802 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573777.271262:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573777.271266:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573777.271270:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616377344:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573777.271300:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616377472:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573777.271328:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573777.271339:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573777/real 1788573777] req@ffff9ada4557d880 x1875449616377472/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573802 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573777.271348:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616377472/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573802 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573777.271354:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573777.271356:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573777.271359:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616377472:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573777.271379:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616377600:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573777.271392:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573777.271399:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573777/real 1788573777] req@ffff9ada4557fb80 x1875449616377600/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573802 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573777.271406:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616377600/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573802 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573777.271412:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573777.271414:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573777.271416:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616377600:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573777.462938:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573777.783056:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573778.486989:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573778.807023:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573779.510967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573779.831036:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573780.534971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573780.855015:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573781.558961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573781.879116:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788573782.390995:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573782.391004:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788573782.391008:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573782.391017:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5694 00000100:00080000:1.0:1788573782.391026:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573782.391034:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788573782.391085:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566dc00 x1875449616377728/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573782.391111:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573782.391114:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788573782.391115:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573782.391118:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5694 00000100:00080000:1.0:1788573782.391120:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573782.391123:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573782.391130:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573782.391149:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f800 x1875449616377856/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573782.391157:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573782.391160:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788573782.391162:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573782.391165:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5694 00000100:00080000:1.0:1788573782.391167:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573782.391170:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573782.391175:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573782.391175:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616377728:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573782.391186:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ed80 x1875449616377984/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788573782.391276:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616377856:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573782.391313:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616377984:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573782.583005:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573782.903022:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573783.606947:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573783.927011:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573784.630950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573784.951012:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1788573785.463193:0:2373:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:1.0:1788573785.463249:0:2373:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1788573785.463264:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:1.0:1788573785.463333:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:1.0:1788573785.463344:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:3.0:1788573785.463445:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573782/real 1788573785] req@ffff9ada4566ed80 x1875449616377984/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573807 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573785.463489:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ed80 x1875449616377984/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573807 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573785.463503:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573785.463509:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573785.463514:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616377984:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573785.463525:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573782/real 1788573785] req@ffff9ada4566dc00 x1875449616377728/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573807 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573785.463539:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566dc00 x1875449616377728/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573807 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573785.463547:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573785.463557:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573785.463561:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616377728:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573785.463569:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573782/real 1788573785] req@ffff9ada4566f800 x1875449616377856/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573807 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573785.463584:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f800 x1875449616377856/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573807 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573785.463593:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573785.463597:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573785.463602:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616377856:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573785.654927:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573785.975058:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573786.678993:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573786.999014:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788573787.510999:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573787.511010:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788573787.511015:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573787.511026:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5700 00000100:00080000:1.0:1788573787.511038:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573787.511046:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788573787.511091:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ea00 x1875449616378112/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573787.511140:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573787.511144:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788573787.511146:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573787.511151:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5700 00000100:00080000:1.0:1788573787.511155:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573787.511159:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573787.511172:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573787.511196:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e300 x1875449616378240/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573787.511206:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573787.511210:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788573787.511213:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573787.511216:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5700 00000100:00080000:1.0:1788573787.511221:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573787.511231:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573787.511236:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573787.511248:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f480 x1875449616378368/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788573787.511383:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616378112:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788573787.511484:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573787.511530:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573787/real 1788573787] req@ffff9ada4566ea00 x1875449616378112/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573812 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573787.511558:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ea00 x1875449616378112/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573812 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573787.511571:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573787.511576:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573787.511581:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616378112:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573787.511588:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616378240:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573787.511617:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573787.511630:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573787/real 1788573787] req@ffff9ada4566e300 x1875449616378240/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573812 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573787.511641:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e300 x1875449616378240/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573812 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573787.511649:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573787.511652:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573787.511655:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616378240:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573787.511661:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616378368:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573787.511687:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573787.511698:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573787/real 1788573787] req@ffff9ada4566f480 x1875449616378368/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573812 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573787.511712:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f480 x1875449616378368/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573812 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573787.511727:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573787.511731:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573787.511735:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616378368:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573787.702987:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573788.023029:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573788.726938:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573789.047035:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573789.750943:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573790.070975:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573790.774951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573791.095050:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573791.798954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573792.118994:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573792.631010:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573792.631021:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573792.631026:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573792.631037:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5705 00000100:00080000:0.0:1788573792.631048:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573792.631056:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573792.631097:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616378496/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573792.631128:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573792.631133:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573792.631135:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573792.631139:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5705 00000100:00080000:0.0:1788573792.631143:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573792.631148:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573792.631158:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573792.631174:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616378624/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788573792.631177:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616378496:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788573792.631195:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573792.631200:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573792.631202:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573792.631207:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5705 00000100:00080000:0.0:1788573792.631212:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573792.631217:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573792.631223:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1788573792.631239:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788573792.631241:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616378752/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788573792.631272:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573792/real 1788573792] req@ffff9ada4557dc00 x1875449616378496/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573817 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573792.631289:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616378496/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573817 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573792.631297:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573792.631300:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573792.631303:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616378496:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573792.631329:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616378624:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573792.631344:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573792.631353:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573792/real 1788573792] req@ffff9ada4557e300 x1875449616378624/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573817 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573792.631361:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616378624/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573817 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573792.631366:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573792.631369:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573792.631371:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616378624:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573792.631374:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616378752:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573792.631384:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573792.631390:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573792/real 1788573792] req@ffff9ada4557ed80 x1875449616378752/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573817 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573792.631398:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616378752/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573817 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573792.631403:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573792.631404:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573792.631415:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616378752:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573792.822937:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573793.143022:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573793.846936:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573794.167035:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573794.870990:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573795.191068:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573795.894974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573796.215052:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573796.918926:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573797.239120:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573797.750948:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573797.750959:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573797.750963:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573797.750974:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5710 00000100:00080000:0.0:1788573797.750985:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573797.750994:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573797.751034:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616378880/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573797.751064:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573797.751069:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573797.751071:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573797.751075:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5710 00000100:00080000:0.0:1788573797.751079:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573797.751083:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573797.751093:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573797.751107:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616379008/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573797.751120:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1788573797.751124:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616378880:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788573797.751134:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573797.751136:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573797.751141:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5710 00000100:00080000:0.0:1788573797.751146:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573797.751151:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573797.751156:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573797.751170:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d500 x1875449616379136/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573797.751185:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573797.751214:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573797/real 1788573797] req@ffff9ada4557ea00 x1875449616378880/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573822 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573797.751229:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616378880/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573822 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573797.751236:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573797.751239:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573797.751242:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616378880:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573797.751261:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616379008:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573797.751276:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573797.751284:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573797/real 1788573797] req@ffff9ada4557e680 x1875449616379008/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573822 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573797.751292:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616379008/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573822 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573797.751297:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573797.751299:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573797.751301:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616379008:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573797.751304:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616379136:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573797.751314:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573797.751320:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573797/real 1788573797] req@ffff9ada4557d500 x1875449616379136/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573822 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573797.751337:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d500 x1875449616379136/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573822 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573797.751349:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573797.751351:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573797.751354:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616379136:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573797.942961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573798.263001:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573798.966950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573799.287054:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573799.990951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573800.311001:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573801.014926:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573801.335004:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573802.038953:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573802.358985:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573802.870951:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573802.870959:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573802.870962:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573802.870971:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5715 00000100:00080000:2.0:1788573802.870981:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573802.870989:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573802.871026:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616379264/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573802.871049:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573802.871052:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573802.871053:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573802.871057:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5715 00000100:00080000:2.0:1788573802.871059:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573802.871062:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573802.871070:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573802.871080:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616379392/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573802.871087:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573802.871089:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573802.871091:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573802.871093:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5715 00000100:00080000:2.0:1788573802.871096:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573802.871098:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573802.871103:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573802.871103:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616379264:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788573802.871115:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616379520/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573802.871168:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573802.871196:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573802/real 1788573802] req@ffff9ada60625f80 x1875449616379264/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573827 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573802.871211:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616379264/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573827 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573802.871219:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573802.871222:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573802.871225:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616379264:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573802.871230:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616379392:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573802.871246:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573802.871254:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573802/real 1788573802] req@ffff9ada60624e00 x1875449616379392/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573827 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573802.871262:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616379392/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573827 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573802.871272:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573802.871274:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573802.871276:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616379392:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573802.871296:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616379520:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573802.871307:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573802.871313:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573802/real 1788573802] req@ffff9ada60627480 x1875449616379520/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573827 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573802.871321:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616379520/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573827 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573802.871326:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573802.871328:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573802.871330:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616379520:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573803.062952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573803.383212:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573804.086997:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573804.406990:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573805.110969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573805.431043:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573806.135004:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573806.455092:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573807.158946:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573807.479058:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573807.991007:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573807.991018:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573807.991023:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573807.991035:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5720 00000100:00080000:2.0:1788573807.991047:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573807.991055:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573807.991100:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616379648/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573807.991129:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573807.991133:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573807.991134:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573807.991137:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5720 00000100:00080000:2.0:1788573807.991139:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573807.991142:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573807.991151:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573807.991160:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616379776/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573807.991168:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573807.991171:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573807.991172:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573807.991175:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5720 00000100:00080000:2.0:1788573807.991177:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573807.991180:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573807.991183:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573807.991188:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616379648:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788573807.991194:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616379904/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573807.991268:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573807.991308:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573807/real 1788573807] req@ffff9ada60625180 x1875449616379648/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573832 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573807.991328:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616379648/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573832 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573807.991342:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573807.991345:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573807.991348:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616379648:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573807.991354:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616379776:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573807.991372:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573807.991382:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573807/real 1788573807] req@ffff9ada60626d80 x1875449616379776/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573832 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573807.991390:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616379776/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573832 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573807.991396:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573807.991398:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573807.991400:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616379776:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573807.991451:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616379904:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573807.991473:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573807.991481:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573807/real 1788573807] req@ffff9ada60627b80 x1875449616379904/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573832 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573807.991489:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616379904/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573832 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573807.991495:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573807.991498:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573807.991500:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616379904:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573808.182972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573808.503046:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573809.206924:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573809.527091:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573810.230948:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573810.551075:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573811.254992:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573811.575011:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573812.278998:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573812.598979:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573813.110928:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573813.110941:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573813.110956:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573813.110971:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5725 00000100:00080000:2.0:1788573813.110986:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573813.110996:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573813.111044:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626300 x1875449616380032/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573813.111072:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573813.111075:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573813.111076:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573813.111079:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5725 00000100:00080000:2.0:1788573813.111082:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573813.111085:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573813.111094:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573813.111104:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627800 x1875449616380160/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573813.111112:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573813.111115:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573813.111116:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573813.111119:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5725 00000100:00080000:2.0:1788573813.111121:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573813.111123:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573813.111126:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573813.111135:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616380032:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788573813.111137:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616380288/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573813.111230:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573813.111277:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573813/real 1788573813] req@ffff9ada60626300 x1875449616380032/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573838 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573813.111302:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626300 x1875449616380032/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573838 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573813.111314:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573813.111319:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573813.111324:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616380032:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573813.111365:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616380160:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573813.111393:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573813.111408:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573813/real 1788573813] req@ffff9ada60627800 x1875449616380160/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573838 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573813.111422:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627800 x1875449616380160/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573838 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573813.111430:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573813.111434:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573813.111438:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616380160:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573813.111477:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616380288:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573813.111505:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573813.111517:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573813/real 1788573813] req@ffff9ada60627b80 x1875449616380288/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573838 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573813.111530:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616380288/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573838 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573813.111539:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573813.111543:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573813.111546:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616380288:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573813.303007:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573813.623004:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573814.326977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573814.646993:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573815.350999:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573815.671019:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573816.374953:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573816.695072:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573817.398974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573817.719045:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573818.230960:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573818.230968:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573818.230972:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573818.230981:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5730 00000100:00080000:3.0:1788573818.230990:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573818.230996:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573818.231030:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1500 x1875449616380416/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573818.231055:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573818.231057:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573818.231059:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573818.231061:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5730 00000100:00080000:3.0:1788573818.231064:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573818.231066:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573818.231075:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573818.231083:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0e00 x1875449616380544/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573818.231090:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573818.231092:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573818.231093:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573818.231095:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5730 00000100:00080000:3.0:1788573818.231097:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573818.231099:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573818.231101:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573818.231107:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2a00 x1875449616380672/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573818.231137:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616380416:192.168.202.113@tcp:250:kworker.0 00000800:00000100:1.0:1788573818.231193:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573818.231222:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573818/real 1788573818] req@ffff9ada604e1500 x1875449616380416/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573843 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573818.231236:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1500 x1875449616380416/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573843 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573818.231271:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573818.231274:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573818.231277:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616380416:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573818.231283:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616380544:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573818.231307:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573818.231316:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573818/real 1788573818] req@ffff9ada604e0e00 x1875449616380544/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573843 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573818.231324:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0e00 x1875449616380544/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573843 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573818.231330:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573818.231332:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573818.231334:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616380544:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573818.231338:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616380672:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573818.231349:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573818.231355:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573818/real 1788573818] req@ffff9ada604e2a00 x1875449616380672/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573843 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573818.231362:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2a00 x1875449616380672/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573843 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573818.231367:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573818.231369:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573818.231370:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616380672:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573818.422948:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573818.742999:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573819.446965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573819.766993:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573820.470951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573820.791043:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573821.494980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573821.815065:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573822.518956:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573822.839018:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573823.350971:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573823.350983:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573823.350996:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573823.351007:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5735 00000100:00080000:2.0:1788573823.351019:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573823.351028:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573823.351068:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616380800/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573823.351090:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573823.351093:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573823.351095:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573823.351097:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5735 00000100:00080000:2.0:1788573823.351101:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573823.351103:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573823.351112:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573823.351123:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616380928/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573823.351132:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616380800:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788573823.351135:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573823.351142:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573823.351143:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573823.351146:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5735 00000100:00080000:2.0:1788573823.351149:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573823.351177:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573823.351182:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573823.351192:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616381056/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573823.351207:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573823.351240:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573823/real 1788573823] req@ffff9ada60626d80 x1875449616380800/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573848 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573823.351253:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616380800/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573848 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573823.351261:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573823.351263:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573823.351266:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616380800:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573823.351285:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616380928:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573823.351300:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573823.351308:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573823/real 1788573823] req@ffff9ada60625180 x1875449616380928/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573848 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573823.351316:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616380928/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573848 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573823.351322:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573823.351324:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573823.351327:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616380928:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573823.351330:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616381056:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573823.351340:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573823.351346:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573823/real 1788573823] req@ffff9ada60627480 x1875449616381056/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573848 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573823.351353:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616381056/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573848 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573823.351358:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573823.351360:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573823.351362:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616381056:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573823.542972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573823.863007:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573824.566953:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573824.887098:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573825.590959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573825.910987:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573826.614931:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573826.935014:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573827.638955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573827.959041:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573828.470973:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573828.470981:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573828.470985:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573828.470993:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5741 00000100:00080000:3.0:1788573828.471002:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573828.471007:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573828.471042:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2a00 x1875449616381184/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573828.471088:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573828.471092:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573828.471094:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573828.471098:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5741 00000100:00080000:3.0:1788573828.471102:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573828.471110:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573828.471119:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573828.471121:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616381184:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573828.471130:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0e00 x1875449616381312/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573828.471139:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573828.471142:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573828.471143:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573828.471146:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5741 00000100:00080000:3.0:1788573828.471148:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573828.471150:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573828.471153:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573828.471160:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1500 x1875449616381440/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573828.471197:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573828.471232:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573828/real 1788573828] req@ffff9ada604e2a00 x1875449616381184/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573853 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573828.471252:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2a00 x1875449616381184/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573853 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573828.471264:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573828.471268:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573828.471272:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616381184:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573828.471294:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616381312:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573828.471317:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573828.471330:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573828/real 1788573828] req@ffff9ada604e0e00 x1875449616381312/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573853 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573828.471343:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0e00 x1875449616381312/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573853 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573828.471352:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573828.471355:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573828.471359:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616381312:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573828.471365:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616381440:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573828.471383:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573828.471394:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573828/real 1788573828] req@ffff9ada604e1500 x1875449616381440/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573853 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573828.471413:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1500 x1875449616381440/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573853 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573828.471421:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573828.471425:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573828.471428:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616381440:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573828.662963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573828.983042:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573829.686936:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573830.007024:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573830.710972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573831.031030:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573831.734937:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573832.055055:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573832.758937:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573833.079034:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573833.590999:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573833.591014:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573833.591019:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573833.591033:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5746 00000100:00080000:3.0:1788573833.591055:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573833.591064:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573833.591115:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2680 x1875449616381568/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573833.591158:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573833.591163:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573833.591165:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573833.591169:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5746 00000100:00080000:3.0:1788573833.591173:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573833.591177:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573833.591188:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573833.591201:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1f80 x1875449616381696/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573833.591213:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1788573833.591215:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616381568:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788573833.591219:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573833.591223:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573833.591228:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5746 00000100:00080000:3.0:1788573833.591232:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573833.591235:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573833.591240:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573833.591251:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3100 x1875449616381824/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573833.591296:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573833.591338:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573833/real 1788573833] req@ffff9ada604e2680 x1875449616381568/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573858 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573833.591355:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2680 x1875449616381568/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573858 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573833.591364:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573833.591367:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573833.591371:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616381568:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573833.591393:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616381696:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573833.591410:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573833.591419:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573833/real 1788573833] req@ffff9ada604e1f80 x1875449616381696/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573858 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573833.591428:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1f80 x1875449616381696/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573858 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573833.591434:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573833.591436:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573833.591438:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616381696:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573833.591442:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616381824:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573833.591453:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573833.591459:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573833/real 1788573833] req@ffff9ada604e3100 x1875449616381824/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573858 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573833.591477:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3100 x1875449616381824/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573858 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573833.591483:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573833.591485:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573833.591487:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616381824:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573833.783033:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573834.103099:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573834.806987:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573835.127073:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573835.830927:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573836.151056:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573836.854968:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573837.175011:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573837.879045:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573838.199335:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573838.711039:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573838.711055:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573838.711059:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573838.711067:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5751 00000100:00080000:0.0:1788573838.711076:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573838.711082:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573838.711117:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557df80 x1875449616381952/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573838.711141:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573838.711144:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573838.711145:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573838.711147:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5751 00000100:00080000:0.0:1788573838.711150:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573838.711152:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573838.711159:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573838.711169:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d180 x1875449616382080/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573838.711185:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573838.711188:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573838.711191:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1788573838.711194:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616381952:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788573838.711198:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5751 00000100:00080000:0.0:1788573838.711205:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573838.711209:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573838.711215:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573838.711231:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d500 x1875449616382208/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573838.711257:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573838.711296:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573838/real 1788573838] req@ffff9ada4557df80 x1875449616381952/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573863 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573838.711310:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557df80 x1875449616381952/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573863 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573838.711318:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573838.711322:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573838.711324:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616381952:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573838.711342:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616382080:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573838.711357:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573838.711365:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573838/real 1788573838] req@ffff9ada4557d180 x1875449616382080/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573863 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573838.711376:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d180 x1875449616382080/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573863 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573838.711384:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573838.711387:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573838.711391:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616382080:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573838.711397:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616382208:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573838.711419:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573838.711430:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573838/real 1788573838] req@ffff9ada4557d500 x1875449616382208/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573863 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573838.711439:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d500 x1875449616382208/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573863 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573838.711449:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573838.711452:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573838.711454:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616382208:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573838.902983:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573839.223019:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573839.927005:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573840.247170:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573840.951000:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573841.271040:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573841.974992:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573842.295128:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573842.998973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573843.318981:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573843.830965:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573843.830973:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573843.830977:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573843.830986:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5756 00000100:00080000:0.0:1788573843.830996:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573843.831002:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573843.831035:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616382336/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573843.831059:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573843.831062:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573843.831067:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573843.831070:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5756 00000100:00080000:0.0:1788573843.831072:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573843.831075:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573843.831083:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573843.831093:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616382464/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573843.831101:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573843.831103:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573843.831110:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1788573843.831113:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616382336:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788573843.831115:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5756 00000100:00080000:0.0:1788573843.831122:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573843.831126:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573843.831130:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573843.831140:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616382592/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573843.831189:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573843.831228:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573843/real 1788573843] req@ffff9ada4557e680 x1875449616382336/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573868 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573843.831245:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616382336/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573868 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573843.831255:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573843.831259:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573843.831262:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616382336:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573843.831269:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616382464:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573843.831291:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573843.831304:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573843/real 1788573843] req@ffff9ada4557ea00 x1875449616382464/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573868 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573843.831316:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616382464/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573868 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573843.831324:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573843.831338:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573843.831346:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616382464:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573843.831368:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616382592:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573843.831386:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573843.831394:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573843/real 1788573843] req@ffff9ada4557ed80 x1875449616382592/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573868 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573843.831402:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616382592/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573868 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573843.831407:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573843.831409:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573843.831411:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616382592:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573844.022966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573844.343041:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573845.046973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573845.367038:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573846.070957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573846.390984:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573847.094982:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573847.415075:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573848.118908:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573848.439048:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573848.950974:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573848.950982:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573848.950986:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573848.950994:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5761 00000100:00080000:0.0:1788573848.951002:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573848.951008:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573848.951043:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616382720/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573848.951068:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573848.951071:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573848.951072:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573848.951075:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5761 00000100:00080000:0.0:1788573848.951078:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573848.951080:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573848.951088:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573848.951096:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616382848/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573848.951104:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573848.951107:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573848.951109:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573848.951112:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5761 00000100:00080000:0.0:1788573848.951114:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1788573848.951117:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616382720:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788573848.951119:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573848.951126:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573848.951134:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616382976/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573848.951191:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616382848:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573848.951227:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616382976:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573849.142989:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573849.463093:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573850.167013:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573850.487038:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573851.190976:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573851.511089:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1788573852.023157:0:2371:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:0.0:1788573852.023207:0:2371:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1788573852.023224:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:0.0:1788573852.023266:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:0.0:1788573852.023273:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:1.0:1788573852.023357:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573848/real 1788573852] req@ffff9ada4557fb80 x1875449616382976/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573873 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573852.023397:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616382976/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573873 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573852.023417:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573852.023422:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573852.023426:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616382976:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573852.023435:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573848/real 1788573852] req@ffff9ada4557e300 x1875449616382720/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573873 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573852.023450:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616382720/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573873 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573852.023457:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573852.023460:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573852.023463:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616382720:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573852.023471:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573848/real 1788573852] req@ffff9ada4557dc00 x1875449616382848/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573873 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573852.023482:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616382848/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573873 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573852.023491:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573852.023495:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573852.023499:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616382848:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573852.215004:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573852.535047:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573853.238985:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573853.559075:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573854.071011:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573854.071023:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573854.071027:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573854.071038:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5766 00000100:00080000:0.0:1788573854.071050:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573854.071059:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573854.071098:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616383104/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573854.071130:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573854.071133:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573854.071134:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573854.071137:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5766 00000100:00080000:0.0:1788573854.071140:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573854.071142:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573854.071151:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573854.071160:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616383232/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573854.071168:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573854.071171:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573854.071172:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573854.071175:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5766 00000100:00080000:0.0:1788573854.071176:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573854.071179:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573854.071182:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573854.071190:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616383104:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788573854.071196:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616383360/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573854.071269:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573854.071307:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573854/real 1788573854] req@ffff9ada4557d880 x1875449616383104/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573879 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573854.071324:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616383104/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573879 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573854.071334:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573854.071339:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573854.071343:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616383104:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573854.071351:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616383232:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573854.071378:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573854.071393:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573854/real 1788573854] req@ffff9ada4557f800 x1875449616383232/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573879 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573854.071406:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616383232/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573879 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573854.071439:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573854.071443:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573854.071446:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616383232:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573854.071478:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616383360:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573854.071502:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573854.071515:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573854/real 1788573854] req@ffff9ada4557ca80 x1875449616383360/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573879 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573854.071527:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616383360/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573879 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573854.071535:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573854.071538:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573854.071541:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616383360:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573854.263005:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573854.583086:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573855.286967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573855.607099:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573856.310999:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573856.631081:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573857.334969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573857.655091:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573858.358973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573858.679058:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573859.190987:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573859.190999:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573859.191010:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573859.191024:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5771 00000100:00080000:0.0:1788573859.191037:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573859.191046:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573859.191092:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616383488/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573859.191115:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573859.191118:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573859.191120:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573859.191124:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5771 00000100:00080000:0.0:1788573859.191126:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573859.191128:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573859.191139:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573859.191150:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616383616/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573859.191157:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573859.191160:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573859.191162:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573859.191165:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5771 00000100:00080000:0.0:1788573859.191167:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573859.191170:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573859.191181:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573859.191181:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616383488:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788573859.191205:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c000 x1875449616383744/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573859.191256:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573859.191295:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573859/real 1788573859] req@ffff9ada4557f480 x1875449616383488/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573884 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573859.191308:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616383488/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573884 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573859.191316:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573859.191319:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573859.191322:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616383488:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573859.191327:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616383616:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573859.191343:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573859.191352:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573859/real 1788573859] req@ffff9ada4557c700 x1875449616383616/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573884 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573859.191360:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616383616/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573884 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573859.191365:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573859.191367:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573859.191369:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616383616:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573859.191387:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616383744:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573859.191399:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573859.191407:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573859/real 1788573859] req@ffff9ada4557c000 x1875449616383744/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573884 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573859.191419:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c000 x1875449616383744/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573884 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573859.191427:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573859.191431:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573859.191435:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616383744:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573859.382989:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573859.703082:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573860.406993:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573860.727070:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573861.430953:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573861.751101:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573862.454980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573862.775045:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573863.478977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573863.799079:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573864.310998:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573864.311009:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573864.311014:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573864.311024:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5776 00000100:00080000:0.0:1788573864.311036:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573864.311046:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573864.311088:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ce00 x1875449616383872/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573864.311114:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573864.311118:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573864.311119:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573864.311123:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5776 00000100:00080000:0.0:1788573864.311127:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573864.311130:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573864.311140:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573864.311154:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c380 x1875449616384000/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573864.311164:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616383872:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788573864.311171:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573864.311176:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573864.311178:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573864.311183:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5776 00000100:00080000:0.0:1788573864.311188:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573864.311191:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573864.311196:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573864.311208:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c000 x1875449616384128/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573864.311225:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573864.311260:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573864/real 1788573864] req@ffff9ada4557ce00 x1875449616383872/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573889 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573864.311275:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ce00 x1875449616383872/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573889 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573864.311284:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573864.311287:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573864.311291:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616383872:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573864.311311:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616384000:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573864.311327:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573864.311335:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573864/real 1788573864] req@ffff9ada4557c380 x1875449616384000/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573889 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573864.311344:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c380 x1875449616384000/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573889 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573864.311349:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573864.311352:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573864.311354:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616384000:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573864.311357:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616384128:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573864.311368:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573864.311373:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573864/real 1788573864] req@ffff9ada4557c000 x1875449616384128/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573889 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573864.311407:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c000 x1875449616384128/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573889 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573864.311412:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573864.311414:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573864.311417:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616384128:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573864.503014:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573864.823009:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573865.526926:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573865.847026:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573866.550985:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573866.871098:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573867.574992:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573867.895073:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573868.598966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573868.919059:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573869.430969:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573869.430981:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573869.430986:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573869.430996:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5781 00000100:00080000:2.0:1788573869.431008:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573869.431016:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573869.431056:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616384256/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573869.431087:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573869.431091:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573869.431094:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573869.431098:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5781 00000100:00080000:2.0:1788573869.431102:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573869.431107:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573869.431117:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573869.431135:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616384384/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573869.431142:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616384256:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788573869.431155:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573869.431159:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573869.431161:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573869.431167:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5781 00000100:00080000:2.0:1788573869.431171:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573869.431175:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573869.431180:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573869.431194:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616384512/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573869.431223:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573869.431273:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573869/real 1788573869] req@ffff9ada60624e00 x1875449616384256/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573894 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573869.431337:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616384256/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573894 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573869.431349:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573869.431353:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573869.431357:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616384256:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573869.431380:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616384384:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573869.431405:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573869.431418:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573869/real 1788573869] req@ffff9ada60625f80 x1875449616384384/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573894 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573869.431433:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616384384/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573894 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573869.431443:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573869.431446:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573869.431450:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616384384:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573869.431457:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616384512:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573869.431476:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573869.431488:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573869/real 1788573869] req@ffff9ada60625c00 x1875449616384512/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573894 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573869.431501:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616384512/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573894 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573869.431510:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573869.431513:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573869.431517:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616384512:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573869.622964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573869.942995:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573870.646968:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573870.967046:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573871.670978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573871.991039:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573872.694945:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573873.014972:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573873.718961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573874.039047:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573874.550971:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573874.550979:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573874.550983:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573874.550991:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5787 00000100:00080000:2.0:1788573874.551000:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573874.551005:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573874.551039:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616384640/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573874.551063:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573874.551066:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573874.551068:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573874.551075:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5787 00000100:00080000:2.0:1788573874.551078:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573874.551081:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573874.551088:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573874.551096:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616384768/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573874.551104:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573874.551107:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573874.551108:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573874.551111:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5787 00000100:00080000:2.0:1788573874.551116:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573874.551120:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573874.551126:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573874.551138:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616384640:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788573874.551140:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616384896/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573874.551223:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573874.551260:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573874/real 1788573874] req@ffff9ada60625880 x1875449616384640/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573899 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573874.551281:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616384640/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573899 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573874.551293:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573874.551298:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573874.551302:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616384640:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573874.551310:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616384768:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573874.551333:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573874.551348:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573874/real 1788573874] req@ffff9ada60624a80 x1875449616384768/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573899 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573874.551361:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616384768/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573899 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573874.551370:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573874.551374:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573874.551378:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616384768:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573874.551403:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616384896:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573874.551427:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573874.551440:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573874/real 1788573874] req@ffff9ada60625500 x1875449616384896/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573899 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573874.551453:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616384896/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573899 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573874.551468:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573874.551471:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573874.551475:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616384896:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573874.742950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573875.063028:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573875.766949:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573876.087065:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573876.790950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573877.111087:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573877.814962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573878.135000:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573878.838968:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573879.158985:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573879.670921:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573879.670930:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573879.670933:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573879.670941:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5792 00000100:00080000:2.0:1788573879.670950:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573879.670956:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573879.670989:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616385024/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573879.671012:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573879.671015:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573879.671017:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573879.671020:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5792 00000100:00080000:2.0:1788573879.671022:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573879.671024:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573879.671031:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573879.671040:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616385152/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573879.671047:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573879.671050:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573879.671051:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573879.671053:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5792 00000100:00080000:2.0:1788573879.671055:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573879.671058:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573879.671061:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573879.671064:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616385024:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788573879.671071:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616385280/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573879.671144:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573879.671179:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573879/real 1788573879] req@ffff9ada60626a00 x1875449616385024/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573904 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573879.671194:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616385024/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573904 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573879.671201:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573879.671204:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573879.671207:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616385024:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573879.671212:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616385152:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573879.671227:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573879.671235:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573879/real 1788573879] req@ffff9ada60624380 x1875449616385152/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573904 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573879.671243:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616385152/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573904 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573879.671252:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573879.671255:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573879.671257:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616385152:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573879.671274:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616385280:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573879.671285:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573879.671291:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573879/real 1788573879] req@ffff9ada60626680 x1875449616385280/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573904 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573879.671298:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616385280/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573904 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573879.671303:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573879.671305:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573879.671307:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616385280:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573879.862976:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573880.182991:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573880.886954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573881.207051:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573881.911014:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573882.231094:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573882.934973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573883.255047:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573883.958973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573884.279095:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573884.790952:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573884.790960:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573884.790963:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573884.790992:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5797 00000100:00080000:2.0:1788573884.791000:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573884.791007:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573884.791048:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627100 x1875449616385408/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573884.791077:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573884.791081:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573884.791083:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573884.791087:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5797 00000100:00080000:2.0:1788573884.791091:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573884.791095:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573884.791104:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573884.791116:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624700 x1875449616385536/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573884.791120:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616385408:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788573884.791130:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573884.791133:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573884.791135:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573884.791138:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5797 00000100:00080000:2.0:1788573884.791141:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573884.791143:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573884.791147:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573884.791157:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616385664/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573884.791187:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573884.791223:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573884/real 1788573884] req@ffff9ada60627100 x1875449616385408/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573909 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573884.791239:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627100 x1875449616385408/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573909 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573884.791247:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573884.791250:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573884.791253:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616385408:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573884.791273:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616385536:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573884.791291:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573884.791300:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573884/real 1788573884] req@ffff9ada60624700 x1875449616385536/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573909 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573884.791308:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624700 x1875449616385536/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573909 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573884.791314:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573884.791316:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573884.791319:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616385536:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573884.791323:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616385664:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573884.791335:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573884.791341:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573884/real 1788573884] req@ffff9ada60626680 x1875449616385664/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573909 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573884.791348:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616385664/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573909 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573884.791353:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573884.791355:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573884.791357:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616385664:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573884.982923:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573885.303048:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573886.006986:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573886.327041:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573887.030938:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573887.351015:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573888.054960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573888.375073:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573889.078987:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573889.399045:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573889.911020:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573889.911031:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573889.911036:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573889.911049:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5802 00000100:00080000:3.0:1788573889.911061:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573889.911070:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573889.911117:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3b80 x1875449616385792/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573889.911148:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573889.911152:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573889.911153:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573889.911157:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5802 00000100:00080000:3.0:1788573889.911159:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573889.911162:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573889.911172:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573889.911185:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1c00 x1875449616385920/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573889.911194:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573889.911197:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573889.911199:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573889.911207:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5802 00000100:00100000:1.0:1788573889.911211:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616385792:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788573889.911212:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573889.911220:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573889.911224:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573889.911239:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0000 x1875449616386048/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573889.911295:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573889.911332:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573889/real 1788573889] req@ffff9ada604e3b80 x1875449616385792/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573914 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573889.911346:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3b80 x1875449616385792/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573914 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573889.911355:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573889.911358:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573889.911360:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616385792:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573889.911380:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616385920:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573889.911396:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573889.911405:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573889/real 1788573889] req@ffff9ada604e1c00 x1875449616385920/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573914 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573889.911413:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1c00 x1875449616385920/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573914 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573889.911419:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573889.911421:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573889.911423:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616385920:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573889.911427:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616386048:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573889.911438:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573889.911443:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573889/real 1788573889] req@ffff9ada604e0000 x1875449616386048/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573914 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573889.911452:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0000 x1875449616386048/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573914 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573889.911457:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573889.911459:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573889.911461:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616386048:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573890.103003:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573890.423053:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573891.126969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573891.447015:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573892.150937:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573892.471050:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573893.174961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573893.495033:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573894.198934:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573894.519023:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573895.031015:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573895.031029:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573895.031035:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573895.031049:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5807 00000100:00080000:3.0:1788573895.031063:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573895.031081:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573895.031131:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2d80 x1875449616386176/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573895.031164:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573895.031169:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573895.031171:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573895.031175:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5807 00000100:00080000:3.0:1788573895.031179:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573895.031183:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573895.031196:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573895.031209:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2300 x1875449616386304/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573895.031220:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573895.031226:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573895.031229:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573895.031235:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5807 00000100:00080000:3.0:1788573895.031239:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1788573895.031250:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616386176:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788573895.031250:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573895.031258:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573895.031272:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0a80 x1875449616386432/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573895.031332:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573895.031385:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573895/real 1788573895] req@ffff9ada604e2d80 x1875449616386176/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573920 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573895.031405:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2d80 x1875449616386176/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573920 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573895.031415:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573895.031418:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573895.031421:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616386176:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573895.031427:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616386304:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573895.031460:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573895.031476:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573895/real 1788573895] req@ffff9ada604e2300 x1875449616386304/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573920 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573895.031490:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2300 x1875449616386304/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573920 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573895.031516:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573895.031522:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573895.031524:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616386304:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573895.031547:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616386432:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573895.031565:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573895.031574:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573895/real 1788573895] req@ffff9ada604e0a80 x1875449616386432/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573920 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573895.031582:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0a80 x1875449616386432/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573920 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573895.031587:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573895.031597:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573895.031600:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616386432:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573895.222968:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573895.543036:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573896.246936:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573896.567029:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573897.270954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573897.591006:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573898.294966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573898.615026:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573899.318989:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573899.638996:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573900.151034:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573900.151043:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573900.151046:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573900.151055:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5812 00000100:00080000:0.0:1788573900.151063:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573900.151070:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573900.151104:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616386560/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573900.151134:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573900.151139:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573900.151141:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573900.151145:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5812 00000100:00080000:0.0:1788573900.151158:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573900.151164:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573900.151175:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573900.151187:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616386688/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573900.151196:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573900.151198:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573900.151200:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573900.151202:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5812 00000100:00080000:0.0:1788573900.151206:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573900.151209:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573900.151213:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573900.151223:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616386816/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573900.151232:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616386560:192.168.202.113@tcp:250:kworker.0 00000800:00000100:1.0:1788573900.151347:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573900.151394:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573900/real 1788573900] req@ffff9ada4557c700 x1875449616386560/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573925 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573900.151411:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616386560/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573925 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573900.151426:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573900.151430:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573900.151433:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616386560:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573900.151440:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616386688:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573900.151460:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573900.151468:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573900/real 1788573900] req@ffff9ada4557f480 x1875449616386688/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573925 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573900.151477:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616386688/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573925 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573900.151483:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573900.151485:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573900.151487:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616386688:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573900.151521:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616386816:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573900.151535:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573900.151546:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573900/real 1788573900] req@ffff9ada4557ca80 x1875449616386816/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573925 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573900.151559:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616386816/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573925 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573900.151568:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573900.151572:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573900.151575:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616386816:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573900.342936:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573900.663096:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573901.367011:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573901.687021:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573902.390940:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573902.711047:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573903.414974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573903.735159:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573904.438924:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573904.759033:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573905.271048:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573905.271058:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573905.271063:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573905.271073:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5817 00000100:00080000:0.0:1788573905.271084:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573905.271092:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573905.271133:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616386944/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573905.271164:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573905.271168:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573905.271170:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573905.271175:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5817 00000100:00080000:0.0:1788573905.271180:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573905.271185:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573905.271195:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573905.271211:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616386944:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788573905.271212:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616387072/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573905.271228:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573905.271232:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573905.271235:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573905.271239:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5817 00000100:00080000:0.0:1788573905.271243:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573905.271248:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573905.271253:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573905.271266:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616387200/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573905.271279:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573905.271317:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573905/real 1788573905] req@ffff9ada4557f800 x1875449616386944/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573930 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573905.271331:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616386944/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573930 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573905.271339:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573905.271342:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573905.271344:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616386944:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573905.271363:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616387072:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573905.271381:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573905.271389:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573905/real 1788573905] req@ffff9ada4557d880 x1875449616387072/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573930 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573905.271397:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616387072/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573930 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573905.271402:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573905.271404:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573905.271406:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616387072:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573905.271412:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616387200:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573905.271423:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573905.271429:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573905/real 1788573905] req@ffff9ada4557dc00 x1875449616387200/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573930 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573905.271436:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616387200/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573930 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573905.271457:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573905.271461:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573905.271465:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616387200:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573905.462958:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573905.782985:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573906.486971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573906.807006:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573907.510944:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573907.831039:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573908.534950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573908.855007:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573909.558953:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573909.879034:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573910.390924:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573910.390934:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573910.390939:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573910.390947:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5822 00000100:00080000:0.0:1788573910.390956:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573910.390963:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573910.391003:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616387328/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573910.391032:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573910.391036:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573910.391038:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573910.391042:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5822 00000100:00080000:0.0:1788573910.391046:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573910.391051:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573910.391062:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573910.391076:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616387456/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573910.391083:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616387328:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788573910.391096:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573910.391101:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573910.391103:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573910.391108:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5822 00000100:00080000:0.0:1788573910.391113:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573910.391117:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573910.391123:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573910.391136:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616387584/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573910.391156:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573910.391194:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573910/real 1788573910] req@ffff9ada4557e300 x1875449616387328/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573935 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573910.391215:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616387328/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573935 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573910.391226:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573910.391231:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573910.391235:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616387328:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573910.391260:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616387456:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573910.391285:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573910.391299:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573910/real 1788573910] req@ffff9ada4557fb80 x1875449616387456/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573935 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573910.391312:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616387456/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573935 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573910.391322:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573910.391325:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573910.391329:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616387456:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573910.391336:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616387584:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573910.391356:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573910.391368:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573910/real 1788573910] req@ffff9ada4557ed80 x1875449616387584/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573935 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573910.391380:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616387584/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573935 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573910.391389:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573910.391393:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573910.391396:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616387584:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573910.582964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573910.903019:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573911.606929:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573911.927048:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573912.631073:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573912.951021:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573913.654936:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573913.975025:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573914.678983:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573914.999045:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788573915.511022:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573915.511029:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788573915.511033:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573915.511042:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5828 00000100:00080000:1.0:1788573915.511051:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573915.511057:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788573915.511091:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ca80 x1875449616387712/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573915.511123:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573915.511126:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788573915.511127:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573915.511130:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5828 00000100:00080000:1.0:1788573915.511133:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573915.511135:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573915.511141:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573915.511150:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d180 x1875449616387840/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573915.511157:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573915.511160:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788573915.511161:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573915.511164:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5828 00000100:00080000:1.0:1788573915.511166:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573915.511168:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573915.511171:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573915.511177:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d500 x1875449616387968/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788573915.511182:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616387712:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573915.511255:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616387840:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573915.511282:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616387968:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788573915.702950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573916.023020:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573916.726980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573917.047052:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788573917.750967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573918.071165:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1788573918.583128:0:2370:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:2.0:1788573918.583170:0:2370:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1788573918.583188:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788573918.583231:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788573918.583242:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:3.0:1788573918.583295:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573915/real 1788573918] req@ffff9ada4566d500 x1875449616387968/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573940 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573918.583323:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d500 x1875449616387968/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573940 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573918.583337:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573918.583342:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573918.583347:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616387968:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573918.583355:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573915/real 1788573918] req@ffff9ada4566ca80 x1875449616387712/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573940 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573918.583377:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ca80 x1875449616387712/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573940 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573918.583386:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573918.583390:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573918.583394:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616387712:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573918.583402:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573915/real 1788573918] req@ffff9ada4566d180 x1875449616387840/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573940 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573918.583414:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d180 x1875449616387840/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573940 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573918.583423:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573918.583427:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573918.583430:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616387840:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573918.774960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573919.094993:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573919.799022:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573920.119022:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573920.631006:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573920.631014:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573920.631018:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573920.631026:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5833 00000100:00080000:2.0:1788573920.631036:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573920.631042:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573920.631072:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616388096/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573920.631098:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573920.631101:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573920.631102:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573920.631105:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5833 00000100:00080000:2.0:1788573920.631107:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573920.631109:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573920.631117:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573920.631126:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616388224/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573920.631135:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573920.631138:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573920.631140:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573920.631144:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5833 00000100:00080000:2.0:1788573920.631149:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573920.631153:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573920.631159:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573920.631174:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616388352/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788573920.631207:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616388096:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788573920.631312:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573920.631363:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573920/real 1788573920] req@ffff9ada60624380 x1875449616388096/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573945 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573920.631387:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616388096/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573945 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573920.631401:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573920.631408:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573920.631412:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616388096:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573920.631422:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616388224:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573920.631451:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573920.631467:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573920/real 1788573920] req@ffff9ada60626a00 x1875449616388224/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573945 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573920.631486:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616388224/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573945 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573920.631496:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573920.631500:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573920.631503:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616388224:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573920.631536:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616388352:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573920.631560:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573920.631571:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573920/real 1788573920] req@ffff9ada60625500 x1875449616388352/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573945 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573920.631584:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616388352/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573945 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573920.631592:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573920.631595:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573920.631599:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616388352:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573920.822992:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573921.143026:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573921.846975:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573922.167014:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573922.871000:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573923.191066:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573923.894950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573924.215024:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573924.918958:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573925.239054:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573925.750991:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573925.750999:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573925.751003:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573925.751012:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5838 00000100:00080000:0.0:1788573925.751021:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573925.751028:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573925.751062:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616388480/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573925.751088:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573925.751090:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573925.751092:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573925.751095:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5838 00000100:00080000:0.0:1788573925.751098:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573925.751100:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573925.751109:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573925.751117:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616388608/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573925.751124:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573925.751127:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573925.751129:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573925.751134:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5838 00000100:00080000:0.0:1788573925.751137:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573925.751141:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573925.751147:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573925.751149:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616388480:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788573925.751161:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d500 x1875449616388736/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573925.751213:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573925.751250:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573925/real 1788573925] req@ffff9ada4557ea00 x1875449616388480/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573950 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573925.751269:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616388480/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573950 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573925.751277:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573925.751281:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573925.751285:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616388480:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573925.751293:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616388608:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573925.751319:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573925.751334:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573925/real 1788573925] req@ffff9ada4557e680 x1875449616388608/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573950 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573925.751348:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616388608/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573950 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573925.751357:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573925.751361:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573925.751364:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616388608:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573925.751391:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616388736:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573925.751411:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573925.751420:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573925/real 1788573925] req@ffff9ada4557d500 x1875449616388736/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573950 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573925.751428:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d500 x1875449616388736/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573950 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573925.751434:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573925.751436:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573925.751438:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616388736:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573925.942934:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573926.263036:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573926.966946:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573927.287066:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573927.990964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573928.311051:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573929.014952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573929.335059:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573930.039000:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573930.359062:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573930.870993:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573930.871001:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573930.871004:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573930.871012:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5843 00000100:00080000:2.0:1788573930.871021:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573930.871027:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573930.871060:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616388864/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573930.871087:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573930.871090:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573930.871091:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573930.871094:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5843 00000100:00080000:2.0:1788573930.871096:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573930.871098:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573930.871106:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573930.871115:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616388992/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573930.871124:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573930.871127:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573930.871128:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573930.871130:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5843 00000100:00080000:2.0:1788573930.871132:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573930.871135:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573930.871139:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573930.871155:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616389120/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788573930.871157:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616388864:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788573930.871261:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573930.871308:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573930/real 1788573930] req@ffff9ada60624a80 x1875449616388864/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573955 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573930.871336:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616388864/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573955 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573930.871346:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573930.871350:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573930.871354:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616388864:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573930.871361:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616388992:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573930.871381:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573930.871390:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573930/real 1788573930] req@ffff9ada60625880 x1875449616388992/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573955 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573930.871398:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616388992/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573955 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573930.871407:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573930.871410:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573930.871413:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616388992:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573930.871445:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616389120:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573930.871469:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573930.871482:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573930/real 1788573930] req@ffff9ada60625c00 x1875449616389120/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573955 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573930.871495:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616389120/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573955 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573930.871501:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573930.871503:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573930.871505:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616389120:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573931.062947:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573931.383112:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573932.087045:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573932.407060:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573933.110942:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573933.431022:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573934.134972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573934.455045:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573935.158951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573935.479070:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788573935.991011:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573935.991022:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788573935.991027:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573935.991037:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5848 00000100:00080000:1.0:1788573935.991048:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573935.991056:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788573935.991094:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f100 x1875449616389248/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573935.991125:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573935.991129:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788573935.991130:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573935.991134:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5848 00000100:00080000:1.0:1788573935.991137:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573935.991142:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573935.991151:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573935.991165:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c380 x1875449616389376/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788573935.991169:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616389248:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788573935.991184:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573935.991188:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788573935.991190:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573935.991194:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5848 00000100:00080000:1.0:1788573935.991198:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573935.991201:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573935.991207:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573935.991217:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c700 x1875449616389504/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573935.991238:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573935.991269:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573935/real 1788573935] req@ffff9ada4566f100 x1875449616389248/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573960 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573935.991291:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f100 x1875449616389248/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573960 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573935.991298:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573935.991301:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573935.991304:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616389248:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573935.991323:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616389376:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573935.991340:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573935.991348:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573935/real 1788573935] req@ffff9ada4566c380 x1875449616389376/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573960 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573935.991355:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c380 x1875449616389376/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573960 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573935.991361:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573935.991362:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573935.991364:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616389376:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573935.991370:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616389504:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573935.991381:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573935.991387:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573935/real 1788573935] req@ffff9ada4566c700 x1875449616389504/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573960 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573935.991399:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c700 x1875449616389504/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573960 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573935.991415:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573935.991419:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573935.991422:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616389504:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573936.182962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573936.502983:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573937.206953:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573937.527073:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573938.230927:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573938.550981:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573939.254986:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573939.575065:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573940.278965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573940.598989:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573941.111015:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573941.111025:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573941.111030:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573941.111041:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5853 00000100:00080000:2.0:1788573941.111050:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573941.111057:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573941.111090:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616389632/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573941.111116:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573941.111119:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573941.111120:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573941.111123:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5853 00000100:00080000:2.0:1788573941.111125:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573941.111128:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573941.111135:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573941.111147:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616389760/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573941.111157:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573941.111160:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573941.111162:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573941.111166:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5853 00000100:00080000:2.0:1788573941.111171:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573941.111176:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573941.111181:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573941.111195:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616389888/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788573941.111220:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616389632:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788573941.111304:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573941.111343:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573941/real 1788573941] req@ffff9ada60625f80 x1875449616389632/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573966 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573941.111361:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616389632/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573966 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573941.111371:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573941.111376:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573941.111380:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616389632:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573941.111385:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616389760:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573941.111401:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573941.111409:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573941/real 1788573941] req@ffff9ada60624e00 x1875449616389760/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573966 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573941.111417:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616389760/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573966 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573941.111422:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573941.111424:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573941.111426:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616389760:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573941.111456:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616389888:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573941.111469:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573941.111476:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573941/real 1788573941] req@ffff9ada60627480 x1875449616389888/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573966 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573941.111484:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616389888/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573966 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573941.111489:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573941.111491:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573941.111493:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616389888:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573941.302971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573941.623059:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573942.326986:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573942.647033:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573943.350950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788573943.671053:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573944.374965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573944.695004:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573945.398940:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573945.719036:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573946.230940:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573946.230948:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573946.230952:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573946.230960:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5858 00000100:00080000:2.0:1788573946.230969:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573946.230975:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573946.231006:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616390016/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573946.231028:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573946.231031:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573946.231033:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573946.231035:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5858 00000100:00080000:2.0:1788573946.231038:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573946.231040:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573946.231047:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573946.231055:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616390144/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573946.231065:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573946.231069:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573946.231071:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573946.231076:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5858 00000100:00080000:2.0:1788573946.231080:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573946.231086:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573946.231088:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616390016:192.168.202.113@tcp:250:kworker.0 00000008:00100000:2.0:1788573946.231092:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573946.231108:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616390272/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573946.231157:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573946.231191:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573946/real 1788573946] req@ffff9ada60625180 x1875449616390016/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573971 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573946.231206:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616390016/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573971 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573946.231213:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573946.231216:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573946.231219:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616390016:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573946.231224:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616390144:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573946.231242:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573946.231250:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573946/real 1788573946] req@ffff9ada60626d80 x1875449616390144/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573971 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573946.231260:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616390144/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573971 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573946.231269:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573946.231272:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573946.231276:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616390144:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573946.231303:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616390272:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573946.231325:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573946.231336:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573946/real 1788573946] req@ffff9ada60627b80 x1875449616390272/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573971 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573946.231349:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616390272/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573971 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573946.231357:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573946.231361:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573946.231365:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616390272:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573946.422993:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788573946.743096:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573947.446979:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573947.767059:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573948.470980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573948.791008:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573949.494986:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788573949.814973:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573950.518986:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573950.839013:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573951.350911:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573951.350922:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573951.350927:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573951.350937:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5863 00000100:00080000:2.0:1788573951.350947:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573951.350955:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573951.350999:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627800 x1875449616390400/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573951.351025:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573951.351028:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573951.351029:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573951.351032:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5863 00000100:00080000:2.0:1788573951.351035:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573951.351037:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573951.351045:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573951.351053:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626300 x1875449616390528/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573951.351060:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573951.351063:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573951.351065:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573951.351067:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5863 00000100:00080000:2.0:1788573951.351069:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573951.351071:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573951.351074:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573951.351081:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616390400:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788573951.351085:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616390656/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573951.351154:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573951.351185:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573951/real 1788573951] req@ffff9ada60627800 x1875449616390400/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573976 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573951.351200:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627800 x1875449616390400/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573976 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573951.351207:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573951.351210:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573951.351213:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616390400:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573951.351219:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616390528:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573951.351236:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573951.351243:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573951/real 1788573951] req@ffff9ada60626300 x1875449616390528/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573976 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573951.351251:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626300 x1875449616390528/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573976 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573951.351256:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573951.351258:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573951.351260:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616390528:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573951.351280:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616390656:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573951.351290:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573951.351297:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573951/real 1788573951] req@ffff9ada60627b80 x1875449616390656/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573976 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573951.351304:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616390656/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573976 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573951.351309:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573951.351311:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573951.351313:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616390656:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573951.542977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573951.863061:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573952.566939:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788573952.887025:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573953.591027:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573953.911057:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573954.614955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573954.935119:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573955.638959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788573955.958961:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573956.470991:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573956.471003:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573956.471008:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573956.471038:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5869 00000100:00080000:3.0:1788573956.471051:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573956.471059:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573956.471104:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1180 x1875449616390784/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573956.471134:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573956.471138:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573956.471139:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573956.471142:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5869 00000100:00080000:3.0:1788573956.471145:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573956.471147:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573956.471157:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788573956.471166:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1880 x1875449616390912/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573956.471174:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573956.471177:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573956.471178:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573956.471181:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5869 00000100:00080000:3.0:1788573956.471183:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573956.471185:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573956.471190:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573956.471199:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616390784:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573956.471199:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3800 x1875449616391040/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573956.471280:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573956.471322:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573956/real 1788573956] req@ffff9ada604e1180 x1875449616390784/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573981 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573956.471337:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1180 x1875449616390784/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573981 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573956.471346:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573956.471349:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573956.471353:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616390784:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573956.471357:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616390912:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573956.471372:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573956.471380:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573956/real 1788573956] req@ffff9ada604e1880 x1875449616390912/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573981 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573956.471388:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1880 x1875449616390912/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573981 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573956.471394:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573956.471395:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573956.471397:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616390912:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573956.471418:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616391040:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573956.471430:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573956.471436:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573956/real 1788573956] req@ffff9ada604e3800 x1875449616391040/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573981 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573956.471444:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3800 x1875449616391040/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573981 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573956.471454:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573956.471456:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573956.471459:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616391040:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573956.662941:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573956.983077:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573957.686954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573958.006996:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573958.710948:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788573959.031031:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573959.734965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573960.054982:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573960.758957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573961.079067:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573961.590990:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573961.591000:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573961.591004:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573961.591015:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5874 00000100:00080000:2.0:1788573961.591026:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573961.591034:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573961.591073:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616391168/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573961.591142:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573961.591146:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573961.591147:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573961.591152:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5874 00000100:00080000:2.0:1788573961.591156:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573961.591160:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573961.591170:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573961.591185:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616391296/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573961.591190:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616391168:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788573961.591218:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573961.591222:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573961.591224:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573961.591228:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5874 00000100:00080000:2.0:1788573961.591233:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573961.591238:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573961.591243:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000800:00000100:1.0:1788573961.591253:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788573961.591260:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616391424/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573961.591290:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573961/real 1788573961] req@ffff9ada60626d80 x1875449616391168/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573986 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573961.591320:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616391168/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573986 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573961.591331:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573961.591335:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573961.591339:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616391168:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573961.591365:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616391296:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573961.591389:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573961.591403:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573961/real 1788573961] req@ffff9ada60625180 x1875449616391296/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573986 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573961.591416:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616391296/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573986 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573961.591425:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573961.591427:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573961.591430:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616391296:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573961.591434:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616391424:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573961.591449:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573961.591457:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573961/real 1788573961] req@ffff9ada60627480 x1875449616391424/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573986 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573961.591466:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616391424/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573986 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573961.591471:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573961.591473:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573961.591475:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616391424:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573961.782955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788573962.102972:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573962.806935:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573963.127027:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573963.830942:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573964.151067:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573964.854936:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788573965.174976:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573965.878966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573966.199037:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573966.710939:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573966.710950:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573966.710955:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573966.710966:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5879 00000100:00080000:2.0:1788573966.710976:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573966.710986:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573966.711025:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616391552/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573966.711065:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573966.711069:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573966.711072:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573966.711076:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5879 00000100:00080000:2.0:1788573966.711080:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573966.711084:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573966.711094:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573966.711107:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616391680/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573966.711118:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573966.711122:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573966.711125:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573966.711130:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5879 00000100:00080000:2.0:1788573966.711134:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573966.711153:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573966.711159:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573966.711162:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616391552:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788573966.711181:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616391808/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573966.711219:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573966.711248:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573966/real 1788573966] req@ffff9ada60624e00 x1875449616391552/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573991 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573966.711262:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616391552/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573991 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573966.711270:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573966.711273:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573966.711276:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616391552:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573966.711280:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616391680:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573966.711294:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573966.711301:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573966/real 1788573966] req@ffff9ada60625f80 x1875449616391680/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573991 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573966.711309:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616391680/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573991 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573966.711314:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573966.711316:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573966.711318:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616391680:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573966.711336:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616391808:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573966.711346:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573966.711352:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573966/real 1788573966] req@ffff9ada60625c00 x1875449616391808/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573991 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573966.711359:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616391808/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573991 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573966.711364:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573966.711366:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573966.711367:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616391808:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573966.902934:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573967.223010:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573967.926960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788573968.247053:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573968.950969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573969.271021:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573969.974978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573970.295017:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573970.998940:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788573971.318980:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788573971.830977:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573971.830985:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788573971.830989:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573971.830998:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5884 00000100:00080000:1.0:1788573971.831007:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573971.831013:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788573971.831048:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ce00 x1875449616391936/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788573971.831080:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573971.831084:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788573971.831086:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573971.831090:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5884 00000100:00080000:1.0:1788573971.831093:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573971.831096:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573971.831105:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573971.831116:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d880 x1875449616392064/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788573971.831124:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616391936:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788573971.831128:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788573971.831133:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788573971.831135:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788573971.831140:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5884 00000100:00080000:1.0:1788573971.831144:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788573971.831148:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788573971.831153:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573971.831167:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e680 x1875449616392192/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788573971.831186:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573971.831246:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573971/real 1788573971] req@ffff9ada4566ce00 x1875449616391936/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573996 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573971.831267:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ce00 x1875449616391936/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788573996 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573971.831279:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573971.831284:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573971.831288:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616391936:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573971.831310:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616392064:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573971.831330:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573971.831344:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573971/real 1788573971] req@ffff9ada4566d880 x1875449616392064/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573996 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573971.831358:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d880 x1875449616392064/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573996 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573971.831368:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573971.831372:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573971.831376:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616392064:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573971.831383:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616392192:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788573971.831403:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573971.831415:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573971/real 1788573971] req@ffff9ada4566e680 x1875449616392192/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573996 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788573971.831429:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e680 x1875449616392192/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788573996 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788573971.831438:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788573971.831440:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788573971.831442:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616392192:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573972.022953:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573972.343030:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573973.046977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573973.367024:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573974.070982:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788573974.390968:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573975.094998:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573975.415068:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573976.118965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573976.439031:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788573976.951007:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573976.951018:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788573976.951023:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573976.951034:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5889 00000100:00080000:3.0:1788573976.951046:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573976.951055:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788573976.951092:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0380 x1875449616392320/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573976.951130:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573976.951135:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788573976.951137:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573976.951141:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5889 00000100:00080000:3.0:1788573976.951146:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573976.951152:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573976.951172:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573976.951173:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616392320:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788573976.951191:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0700 x1875449616392448/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788573976.951206:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788573976.951210:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788573976.951213:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788573976.951217:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5889 00000100:00080000:3.0:1788573976.951222:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788573976.951227:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788573976.951248:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000800:00000100:1.0:1788573976.951256:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788573976.951265:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3480 x1875449616392576/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573976.951297:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573976/real 1788573976] req@ffff9ada604e0380 x1875449616392320/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574001 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573976.951313:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0380 x1875449616392320/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574001 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573976.951322:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573976.951325:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573976.951328:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616392320:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573976.951347:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616392448:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573976.951363:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573976.951372:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573976/real 1788573976] req@ffff9ada604e0700 x1875449616392448/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574001 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573976.951380:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0700 x1875449616392448/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574001 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573976.951386:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573976.951389:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573976.951393:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616392448:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573976.951402:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616392576:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573976.951423:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573976.951436:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573976/real 1788573976] req@ffff9ada604e3480 x1875449616392576/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574001 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573976.951449:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3480 x1875449616392576/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574001 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573976.951456:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573976.951458:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573976.951460:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616392576:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573977.142988:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788573977.462945:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573978.166987:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573978.487056:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573979.190989:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573979.511015:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573980.214944:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788573980.534998:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573981.238961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573981.559020:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788573982.070993:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573982.071001:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788573982.071005:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573982.071014:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5894 00000100:00080000:2.0:1788573982.071023:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573982.071030:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788573982.071060:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616392704/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573982.071084:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573982.071087:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788573982.071088:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573982.071091:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5894 00000100:00080000:2.0:1788573982.071093:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573982.071095:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573982.071103:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573982.071110:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616392832/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788573982.071117:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788573982.071120:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788573982.071121:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788573982.071124:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5894 00000100:00080000:2.0:1788573982.071126:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788573982.071129:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788573982.071132:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788573982.071138:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616392960/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788573982.071154:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616392704:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573982.071267:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616392832:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573982.071309:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616392960:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573982.262984:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573982.583073:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573983.286976:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788573983.607025:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573984.310944:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573984.631037:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1788573985.143149:0:2372:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:0.0:1788573985.143196:0:2372:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1788573985.143214:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:0.0:1788573985.143293:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:0.0:1788573985.143305:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:1.0:1788573985.143411:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573982/real 1788573985] req@ffff9ada60625500 x1875449616392960/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574007 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573985.143439:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616392960/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574007 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573985.143458:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573985.143463:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573985.143469:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616392960:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573985.143480:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573982/real 1788573985] req@ffff9ada60625880 x1875449616392704/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574007 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573985.143493:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616392704/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574007 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573985.143503:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573985.143507:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573985.143510:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616392704:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573985.143516:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573982/real 1788573985] req@ffff9ada60624a80 x1875449616392832/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574007 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573985.143528:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616392832/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574007 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573985.143536:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573985.143540:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573985.143543:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616392832:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788573985.334972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573985.655037:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788573986.358970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788573986.678983:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573987.191031:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573987.191038:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573987.191042:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573987.191050:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5899 00000100:00080000:0.0:1788573987.191058:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573987.191064:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573987.191093:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d180 x1875449616393088/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573987.191116:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573987.191120:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573987.191122:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573987.191126:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5899 00000100:00080000:0.0:1788573987.191130:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573987.191134:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573987.191143:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573987.191158:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557df80 x1875449616393216/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573987.191168:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573987.191172:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573987.191175:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1788573987.191181:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616393088:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788573987.191182:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5899 00000100:00080000:0.0:1788573987.191188:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573987.191194:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573987.191200:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573987.191216:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d500 x1875449616393344/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573987.191247:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573987.191311:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573987/real 1788573987] req@ffff9ada4557d180 x1875449616393088/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574012 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573987.191331:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d180 x1875449616393088/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574012 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573987.191342:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573987.191347:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573987.191350:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616393088:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573987.191359:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616393216:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573987.191384:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573987.191398:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573987/real 1788573987] req@ffff9ada4557df80 x1875449616393216/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574012 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573987.191410:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557df80 x1875449616393216/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574012 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573987.191418:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573987.191422:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573987.191425:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616393216:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573987.191452:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616393344:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573987.191472:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573987.191483:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573987/real 1788573987] req@ffff9ada4557d500 x1875449616393344/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574012 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573987.191495:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d500 x1875449616393344/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574012 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573987.191508:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573987.191511:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573987.191515:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616393344:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573987.382986:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573987.703078:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573988.407025:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573988.727062:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573989.430952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788573989.751015:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573990.454978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573990.775012:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573991.478962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573991.798984:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573992.310981:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573992.310993:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573992.310997:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573992.311007:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5904 00000100:00080000:0.0:1788573992.311018:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573992.311024:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573992.311059:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616393472/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573992.311086:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573992.311088:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573992.311090:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573992.311092:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5904 00000100:00080000:0.0:1788573992.311095:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573992.311097:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573992.311105:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573992.311114:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616393600/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573992.311126:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573992.311128:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573992.311130:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573992.311132:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5904 00000100:00080000:0.0:1788573992.311135:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1788573992.311139:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616393472:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788573992.311140:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573992.311147:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573992.311154:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616393728/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573992.311200:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573992.311231:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573992/real 1788573992] req@ffff9ada4557e680 x1875449616393472/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574017 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573992.311244:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616393472/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574017 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573992.311252:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573992.311255:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573992.311258:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616393472:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573992.311263:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616393600:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573992.311277:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573992.311301:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573992/real 1788573992] req@ffff9ada4557ea00 x1875449616393600/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574017 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573992.311314:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616393600/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574017 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573992.311320:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573992.311323:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573992.311325:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616393600:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573992.311344:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616393728:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573992.311366:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573992.311374:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573992/real 1788573992] req@ffff9ada4557ed80 x1875449616393728/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574017 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573992.311382:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616393728/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574017 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573992.311388:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573992.311390:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573992.311392:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616393728:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573992.502971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788573992.822987:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573993.526923:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573993.847028:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573994.551048:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573994.871008:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573995.574991:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788573995.895068:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573996.598986:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573996.919161:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788573997.430946:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573997.430956:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788573997.430961:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573997.430972:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5909 00000100:00080000:0.0:1788573997.430984:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573997.430992:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788573997.431044:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616393856/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573997.431070:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573997.431073:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788573997.431074:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573997.431077:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5909 00000100:00080000:0.0:1788573997.431080:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573997.431083:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573997.431093:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788573997.431102:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616393984/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788573997.431110:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788573997.431113:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788573997.431115:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788573997.431117:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5909 00000100:00080000:0.0:1788573997.431119:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788573997.431122:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788573997.431126:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788573997.431128:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616393856:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788573997.431135:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616394112/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788573997.431204:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573997.431245:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573997/real 1788573997] req@ffff9ada4557fb80 x1875449616393856/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574022 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573997.431262:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616393856/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574022 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573997.431270:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573997.431274:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573997.431277:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616393856:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788573997.431283:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616393984:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573997.431298:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573997.431322:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573997/real 1788573997] req@ffff9ada4557e300 x1875449616393984/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574022 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573997.431332:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616393984/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574022 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573997.431338:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573997.431340:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573997.431342:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616393984:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573997.431365:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616394112:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788573997.431380:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788573997.431388:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788573997/real 1788573997] req@ffff9ada4557dc00 x1875449616394112/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574022 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788573997.431400:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616394112/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574022 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788573997.431406:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788573997.431407:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788573997.431409:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616394112:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788573997.622960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788573997.943073:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573998.646967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788573998.966975:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788573999.670968:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788573999.991049:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574000.695066:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574001.015016:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574001.718957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574002.038987:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574002.550989:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574002.551002:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574002.551006:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574002.551018:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5915 00000100:00080000:0.0:1788574002.551031:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574002.551038:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574002.551085:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616394240/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574002.551114:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574002.551117:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574002.551119:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574002.551122:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5915 00000100:00080000:0.0:1788574002.551125:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574002.551127:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574002.551138:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574002.551148:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616394368/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574002.551156:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574002.551160:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574002.551162:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574002.551169:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5915 00000100:00080000:0.0:1788574002.551173:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1788574002.551175:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616394240:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788574002.551177:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574002.551185:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574002.551199:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616394496/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788574002.551280:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574002.551341:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574002/real 1788574002] req@ffff9ada4557d880 x1875449616394240/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574027 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574002.551359:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616394240/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574027 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574002.551368:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574002.551372:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574002.551375:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616394240:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574002.551400:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616394368:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574002.551426:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574002.551440:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574002/real 1788574002] req@ffff9ada4557f800 x1875449616394368/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574027 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574002.551454:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616394368/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574027 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574002.551464:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574002.551467:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574002.551470:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616394368:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574002.551474:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616394496:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574002.551491:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574002.551499:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574002/real 1788574002] req@ffff9ada4557ca80 x1875449616394496/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574027 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574002.551507:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616394496/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574027 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574002.551513:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574002.551515:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574002.551517:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616394496:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574002.743003:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574003.063068:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574003.766955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574004.087004:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574004.790952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574005.111007:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574005.814959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574006.135034:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574006.838970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574007.159054:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788574007.670966:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574007.670973:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788574007.670976:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574007.670984:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5920 00000100:00080000:1.0:1788574007.670992:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574007.670998:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574007.671033:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e680 x1875449616394624/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574007.671070:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574007.671075:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788574007.671077:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574007.671081:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5920 00000100:00080000:1.0:1788574007.671085:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574007.671089:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574007.671099:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574007.671115:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616394624:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574007.671116:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d880 x1875449616394752/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574007.671133:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574007.671135:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788574007.671137:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574007.671141:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5920 00000100:00080000:1.0:1788574007.671145:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574007.671148:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574007.671154:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574007.671166:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ce00 x1875449616394880/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574007.671188:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574007.671219:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574007/real 1788574007] req@ffff9ada4566e680 x1875449616394624/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574032 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574007.671234:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e680 x1875449616394624/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574032 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574007.671242:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574007.671245:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574007.671248:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616394624:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574007.671269:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616394752:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574007.671292:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574007.671306:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574007/real 1788574007] req@ffff9ada4566d880 x1875449616394752/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574032 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574007.671320:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d880 x1875449616394752/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574032 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574007.671327:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574007.671329:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574007.671331:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616394752:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574007.671335:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616394880:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574007.671368:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574007.671377:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574007/real 1788574007] req@ffff9ada4566ce00 x1875449616394880/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574032 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574007.671385:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ce00 x1875449616394880/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574032 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574007.671390:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574007.671393:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574007.671395:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616394880:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574007.862987:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574008.183012:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574008.886940:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574009.207062:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574009.910963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574010.230975:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574010.934962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574011.255022:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574011.958979:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574012.278991:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788574012.790990:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574012.791005:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788574012.791011:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574012.791025:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5925 00000100:00080000:1.0:1788574012.791038:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574012.791048:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574012.791102:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c700 x1875449616395008/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574012.791142:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574012.791147:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788574012.791149:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574012.791153:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5925 00000100:00080000:1.0:1788574012.791158:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574012.791162:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574012.791175:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574012.791191:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c380 x1875449616395136/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574012.791204:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574012.791209:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788574012.791211:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574012.791216:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5925 00000100:00080000:1.0:1788574012.791220:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574012.791227:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574012.791231:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616395008:192.168.202.113@tcp:250:kworker.0 00000008:00100000:1.0:1788574012.791234:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574012.791250:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f100 x1875449616395264/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574012.791307:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574012.791388:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574012/real 1788574012] req@ffff9ada4566c700 x1875449616395008/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574037 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574012.791413:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c700 x1875449616395008/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574037 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574012.791434:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574012.791438:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574012.791443:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616395008:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574012.791452:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616395136:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574012.791475:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574012.791487:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574012/real 1788574012] req@ffff9ada4566c380 x1875449616395136/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574037 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574012.791496:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c380 x1875449616395136/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574037 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574012.791502:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574012.791517:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574012.791524:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616395136:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574012.791556:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616395264:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574012.791580:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574012.791593:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574012/real 1788574012] req@ffff9ada4566f100 x1875449616395264/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574037 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574012.791606:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f100 x1875449616395264/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574037 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574012.791612:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574012.791615:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574012.791617:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616395264:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574012.982983:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574013.303017:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574014.006973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574014.326966:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574015.030971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574015.351034:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574016.054943:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574016.374996:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574017.078979:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574017.399003:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574017.911037:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574017.911047:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574017.911052:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574017.911063:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5930 00000100:00080000:2.0:1788574017.911075:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574017.911083:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574017.911128:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616395392/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574017.911170:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574017.911175:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574017.911176:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574017.911181:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5930 00000100:00080000:2.0:1788574017.911184:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574017.911188:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574017.911199:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574017.911212:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616395520/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574017.911221:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574017.911224:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574017.911225:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574017.911230:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5930 00000100:00100000:3.0:1788574017.911232:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616395392:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788574017.911233:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574017.911240:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574017.911245:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574017.911252:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616395648/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574017.911311:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574017.911350:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574017/real 1788574017] req@ffff9ada60626a00 x1875449616395392/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574042 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574017.911367:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616395392/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574042 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574017.911398:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574017.911402:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574017.911405:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616395392:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574017.911429:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616395520:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574017.911446:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574017.911455:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574017/real 1788574017] req@ffff9ada60624380 x1875449616395520/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574042 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574017.911464:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616395520/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574042 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574017.911470:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574017.911472:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574017.911474:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616395520:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574017.911478:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616395648:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574017.911488:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574017.911494:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574017/real 1788574017] req@ffff9ada60626680 x1875449616395648/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574042 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574017.911502:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616395648/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574042 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574017.911507:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574017.911509:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574017.911511:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616395648:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574018.102993:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574018.423009:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574019.126965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574019.447029:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574020.151030:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574020.470991:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574021.174946:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574021.495058:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574022.198950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574022.519035:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788574023.030998:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574023.031006:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788574023.031009:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574023.031018:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5935 00000100:00080000:3.0:1788574023.031027:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574023.031032:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574023.031064:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3480 x1875449616395776/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574023.031086:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574023.031089:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788574023.031090:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574023.031093:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5935 00000100:00080000:3.0:1788574023.031096:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574023.031098:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574023.031105:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574023.031118:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0700 x1875449616395904/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788574023.031131:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616395776:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788574023.031141:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574023.031146:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788574023.031148:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574023.031176:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5935 00000100:00080000:3.0:1788574023.031181:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574023.031184:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574023.031189:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574023.031201:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0380 x1875449616396032/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788574023.031205:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574023.031243:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574023/real 1788574023] req@ffff9ada604e3480 x1875449616395776/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574048 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574023.031257:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3480 x1875449616395776/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574048 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574023.031265:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574023.031268:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574023.031271:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616395776:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574023.031290:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616395904:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574023.031314:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574023.031327:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574023/real 1788574023] req@ffff9ada604e0700 x1875449616395904/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574048 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574023.031342:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0700 x1875449616395904/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574048 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574023.031349:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574023.031352:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574023.031354:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616395904:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574023.031359:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616396032:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574023.031374:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574023.031382:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574023/real 1788574023] req@ffff9ada604e0380 x1875449616396032/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574048 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574023.031389:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0380 x1875449616396032/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574048 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574023.031410:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574023.031413:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574023.031415:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616396032:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574023.222942:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574023.542984:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574024.246939:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574024.567060:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574025.270940:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574025.590984:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574026.294951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574026.615021:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574027.318944:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574027.638986:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788574028.150968:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574028.150977:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788574028.150981:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574028.150992:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5940 00000100:00080000:3.0:1788574028.151004:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574028.151011:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574028.151057:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3800 x1875449616396160/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574028.151081:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574028.151083:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788574028.151085:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574028.151088:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5940 00000100:00080000:3.0:1788574028.151091:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574028.151093:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574028.151102:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574028.151112:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1880 x1875449616396288/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574028.151120:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574028.151124:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:1.0:1788574028.151126:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616396160:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788574028.151126:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574028.151134:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5940 00000100:00080000:3.0:1788574028.151136:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574028.151139:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574028.151141:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574028.151150:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1180 x1875449616396416/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788574028.151187:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574028.151218:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574028/real 1788574028] req@ffff9ada604e3800 x1875449616396160/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574053 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574028.151233:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3800 x1875449616396160/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574053 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574028.151241:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574028.151243:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574028.151247:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616396160:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574028.151291:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616396288:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574028.151308:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574028.151316:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574028/real 1788574028] req@ffff9ada604e1880 x1875449616396288/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574053 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574028.151325:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1880 x1875449616396288/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574053 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574028.151331:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574028.151333:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574028.151335:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616396288:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574028.151339:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616396416:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574028.151349:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574028.151355:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574028/real 1788574028] req@ffff9ada604e1180 x1875449616396416/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574053 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574028.151363:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1180 x1875449616396416/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574053 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574028.151368:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574028.151370:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574028.151371:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616396416:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574028.342939:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574028.663037:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574029.366959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574029.686957:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574030.390934:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574030.711008:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574031.414955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574031.735048:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574032.438968:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574032.759119:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574033.270981:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574033.270990:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574033.270994:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574033.271003:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5945 00000100:00080000:2.0:1788574033.271020:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574033.271027:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574033.271060:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624700 x1875449616396544/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574033.271082:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574033.271085:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574033.271087:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574033.271089:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5945 00000100:00080000:2.0:1788574033.271092:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574033.271094:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574033.271101:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574033.271111:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627100 x1875449616396672/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574033.271119:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574033.271121:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574033.271123:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574033.271126:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5945 00000100:00080000:2.0:1788574033.271128:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574033.271132:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574033.271134:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616396544:192.168.202.113@tcp:250:kworker.0 00000008:00100000:2.0:1788574033.271136:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574033.271148:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616396800/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788574033.271193:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574033.271222:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574033/real 1788574033] req@ffff9ada60624700 x1875449616396544/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574058 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574033.271235:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624700 x1875449616396544/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574058 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574033.271242:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574033.271245:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574033.271248:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616396544:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574033.271253:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616396672:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574033.271266:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574033.271274:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574033/real 1788574033] req@ffff9ada60627100 x1875449616396672/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574058 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574033.271282:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627100 x1875449616396672/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574058 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574033.271288:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574033.271289:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574033.271291:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616396672:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574033.271315:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616396800:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574033.271336:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574033.271348:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574033/real 1788574033] req@ffff9ada60626680 x1875449616396800/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574058 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574033.271360:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616396800/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574058 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574033.271368:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574033.271371:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574033.271374:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616396800:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574033.462981:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574033.783000:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574034.486950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574034.807039:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574035.510950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574035.830942:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574036.534952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574036.854980:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574037.559006:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574037.879043:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574038.390957:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574038.390969:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574038.390974:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574038.390984:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5950 00000100:00080000:2.0:1788574038.390994:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574038.391002:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574038.391037:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616396928/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574038.391066:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574038.391070:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574038.391072:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574038.391076:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5950 00000100:00080000:2.0:1788574038.391080:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574038.391084:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574038.391094:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574038.391108:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616397056/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574038.391122:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574038.391126:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574038.391129:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1788574038.391137:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616396928:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788574038.391138:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5950 00000100:00080000:2.0:1788574038.391147:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574038.391159:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574038.391165:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574038.391182:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616397184/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788574038.391251:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574038.391301:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574038/real 1788574038] req@ffff9ada60624380 x1875449616396928/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574063 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574038.391326:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616396928/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574063 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574038.391340:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574038.391345:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574038.391350:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616396928:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574038.391381:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616397056:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574038.391407:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574038.391421:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574038/real 1788574038] req@ffff9ada60626a00 x1875449616397056/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574063 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574038.391435:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616397056/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574063 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574038.391445:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574038.391464:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574038.391469:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616397056:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574038.391477:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616397184:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574038.391506:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574038.391520:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574038/real 1788574038] req@ffff9ada60624a80 x1875449616397184/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574063 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574038.391535:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616397184/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574063 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574038.391552:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574038.391555:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574038.391559:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616397184:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574038.582958:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574038.902985:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574039.606964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574039.927113:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574040.630939:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574040.951022:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574041.654988:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574041.974991:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574042.678954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574042.999025:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574043.510991:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574043.510999:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574043.511003:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574043.511011:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5956 00000100:00080000:0.0:1788574043.511021:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574043.511027:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574043.511059:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616397312/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574043.511082:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574043.511085:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574043.511087:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574043.511091:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5956 00000100:00080000:0.0:1788574043.511095:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574043.511100:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574043.511109:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574043.511123:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616397440/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574043.511136:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574043.511140:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574043.511142:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574043.511146:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5956 00000100:00080000:0.0:1788574043.511149:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574043.511153:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574043.511156:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574043.511168:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c000 x1875449616397568/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788574043.511187:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616397312:192.168.202.113@tcp:250:kworker.0 00000800:00000100:1.0:1788574043.511298:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574043.511354:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574043/real 1788574043] req@ffff9ada4557f480 x1875449616397312/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574068 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574043.511390:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616397312/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574068 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574043.511404:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574043.511437:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574043.511443:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616397312:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574043.511453:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616397440:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574043.511483:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574043.511499:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574043/real 1788574043] req@ffff9ada4557c700 x1875449616397440/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574068 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574043.511513:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616397440/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574068 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574043.511523:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574043.511528:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574043.511531:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616397440:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574043.511576:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616397568:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574043.511603:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574043.511617:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574043/real 1788574043] req@ffff9ada4557c000 x1875449616397568/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574068 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574043.511630:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c000 x1875449616397568/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574068 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574043.511640:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574043.511644:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574043.511648:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616397568:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574043.702972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574044.023054:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574044.726959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574045.046969:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574045.750970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574046.071046:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574046.774973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574047.095007:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574047.798933:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574048.119030:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788574048.630973:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574048.630983:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788574048.630988:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574048.630998:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5961 00000100:00080000:3.0:1788574048.631008:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574048.631014:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574048.631050:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0a80 x1875449616397696/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574048.631079:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574048.631083:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788574048.631085:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574048.631090:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5961 00000100:00080000:3.0:1788574048.631093:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574048.631103:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574048.631112:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574048.631124:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2300 x1875449616397824/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574048.631132:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574048.631135:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788574048.631136:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574048.631141:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5961 00000100:00100000:1.0:1788574048.631145:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616397696:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788574048.631145:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574048.631150:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574048.631157:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574048.631167:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2d80 x1875449616397952/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788574048.631245:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616397824:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574048.631288:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616397952:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574048.822956:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574049.143053:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574049.846963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574050.167073:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574050.870986:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574051.191008:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1788574051.703138:0:2373:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:2.0:1788574051.703210:0:2373:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1788574051.703227:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788574051.703269:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788574051.703280:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:1.0:1788574051.703329:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574048/real 1788574051] req@ffff9ada604e2d80 x1875449616397952/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574073 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574051.703367:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2d80 x1875449616397952/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574073 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574051.703381:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574051.703386:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574051.703390:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616397952:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574051.703399:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574048/real 1788574051] req@ffff9ada604e0a80 x1875449616397696/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574073 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574051.703412:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0a80 x1875449616397696/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574073 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574051.703420:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574051.703423:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574051.703427:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616397696:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574051.703434:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574048/real 1788574051] req@ffff9ada604e2300 x1875449616397824/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574073 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574051.703447:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2300 x1875449616397824/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574073 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574051.703456:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574051.703460:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574051.703464:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616397824:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574051.894977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574052.215035:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574052.918961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574053.238977:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788574053.750947:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574053.750966:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788574053.750971:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574053.750982:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5966 00000100:00080000:3.0:1788574053.750993:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574053.751000:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574053.751031:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0000 x1875449616398080/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574053.751054:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574053.751058:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788574053.751059:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574053.751062:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5966 00000100:00080000:3.0:1788574053.751064:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574053.751067:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574053.751075:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574053.751084:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1c00 x1875449616398208/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574053.751092:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574053.751095:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788574053.751096:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1788574053.751102:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616398080:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788574053.751104:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5966 00000100:00080000:3.0:1788574053.751112:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574053.751117:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574053.751122:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574053.751136:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3b80 x1875449616398336/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788574053.751175:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574053.751210:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574053/real 1788574053] req@ffff9ada604e0000 x1875449616398080/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574078 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574053.751230:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0000 x1875449616398080/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574078 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574053.751242:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574053.751246:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574053.751250:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616398080:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574053.751277:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616398208:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574053.751305:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574053.751321:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574053/real 1788574053] req@ffff9ada604e1c00 x1875449616398208/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574078 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574053.751350:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1c00 x1875449616398208/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574078 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574053.751360:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574053.751364:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574053.751380:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616398208:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574053.751388:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616398336:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574053.751413:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574053.751443:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574053/real 1788574053] req@ffff9ada604e3b80 x1875449616398336/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574078 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574053.751457:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3b80 x1875449616398336/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574078 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574053.751474:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574053.751478:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574053.751481:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616398336:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574053.942960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574054.262970:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574054.966951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574055.287031:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574055.990980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574056.310998:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574057.014954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574057.334950:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574058.038957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574058.358990:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788574058.870990:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574058.871001:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788574058.871005:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574058.871021:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5971 00000100:00080000:3.0:1788574058.871033:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574058.871041:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574058.871084:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3100 x1875449616398464/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574058.871110:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574058.871113:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788574058.871115:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574058.871118:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5971 00000100:00080000:3.0:1788574058.871120:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574058.871122:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574058.871136:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574058.871146:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1f80 x1875449616398592/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574058.871154:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574058.871156:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788574058.871158:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574058.871160:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5971 00000100:00080000:3.0:1788574058.871162:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574058.871165:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574058.871170:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616398464:192.168.202.113@tcp:250:kworker.0 00000008:00100000:3.0:1788574058.871171:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574058.871187:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2680 x1875449616398720/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788574058.871248:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574058.871290:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574058/real 1788574058] req@ffff9ada604e3100 x1875449616398464/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574083 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574058.871322:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3100 x1875449616398464/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574083 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574058.871332:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574058.871336:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574058.871339:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616398464:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574058.871345:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616398592:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574058.871365:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574058.871375:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574058/real 1788574058] req@ffff9ada604e1f80 x1875449616398592/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574083 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574058.871384:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1f80 x1875449616398592/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574083 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574058.871390:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574058.871392:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574058.871394:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616398592:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574058.871418:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616398720:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574058.871430:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574058.871436:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574058/real 1788574058] req@ffff9ada604e2680 x1875449616398720/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574083 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574058.871443:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2680 x1875449616398720/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574083 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574058.871449:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574058.871451:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574058.871453:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616398720:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574059.062962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574059.382989:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574060.086972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574060.406990:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574061.110913:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574061.431069:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574062.135035:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574062.455007:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574063.158969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574063.479005:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574063.991007:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574063.991022:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574063.991027:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574063.991042:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5976 00000100:00080000:0.0:1788574063.991071:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574063.991080:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574063.991145:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c380 x1875449616398848/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574063.991180:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574063.991185:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574063.991187:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574063.991192:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5976 00000100:00080000:0.0:1788574063.991196:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574063.991201:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574063.991213:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574063.991227:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ce00 x1875449616398976/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574063.991239:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574063.991245:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574063.991247:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574063.991269:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5976 00000100:00080000:0.0:1788574063.991275:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574063.991281:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574063.991302:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574063.991307:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616398848:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574063.991321:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c000 x1875449616399104/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788574063.991399:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574063.991441:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574063/real 1788574063] req@ffff9ada4557c380 x1875449616398848/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574088 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574063.991458:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c380 x1875449616398848/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574088 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574063.991468:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574063.991471:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574063.991474:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616398848:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574063.991480:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616398976:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574063.991497:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574063.991505:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574063/real 1788574063] req@ffff9ada4557ce00 x1875449616398976/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574088 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574063.991514:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ce00 x1875449616398976/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574088 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574063.991519:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574063.991521:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574063.991523:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616398976:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574063.991546:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616399104:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574063.991557:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574063.991580:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574063/real 1788574063] req@ffff9ada4557c000 x1875449616399104/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574088 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574063.991595:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c000 x1875449616399104/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574088 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574063.991601:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574063.991603:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574063.991605:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616399104:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574064.182990:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574064.503012:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574065.206960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574065.527069:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574066.230960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574066.550977:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574067.254984:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574067.575114:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574068.278942:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574068.599054:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788574069.111019:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574069.111027:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788574069.111032:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574069.111041:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5981 00000100:00080000:3.0:1788574069.111052:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574069.111059:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574069.111097:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1500 x1875449616399232/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574069.111125:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574069.111128:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788574069.111129:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574069.111133:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5981 00000100:00080000:3.0:1788574069.111136:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574069.111140:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574069.111148:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574069.111160:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0e00 x1875449616399360/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788574069.111173:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616399232:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788574069.111173:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574069.111186:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788574069.111188:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574069.111192:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5981 00000100:00080000:3.0:1788574069.111194:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574069.111196:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574069.111200:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574069.111212:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2a00 x1875449616399488/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788574069.111292:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574069.111340:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574069/real 1788574069] req@ffff9ada604e1500 x1875449616399232/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574094 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574069.111364:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1500 x1875449616399232/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574094 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574069.111374:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574069.111378:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574069.111381:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616399232:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574069.111407:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616399360:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574069.111427:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574069.111437:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574069/real 1788574069] req@ffff9ada604e0e00 x1875449616399360/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574094 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574069.111445:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0e00 x1875449616399360/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574094 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574069.111451:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574069.111454:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574069.111456:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616399360:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574069.111461:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616399488:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574069.111473:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574069.111479:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574069/real 1788574069] req@ffff9ada604e2a00 x1875449616399488/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574094 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574069.111486:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2a00 x1875449616399488/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574094 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574069.111492:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574069.111494:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574069.111495:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616399488:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574069.302986:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574069.622986:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574070.326983:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574070.647045:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574071.350926:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574071.671025:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574072.374971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574072.694973:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574073.398961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574073.719006:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788574074.231031:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574074.231045:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788574074.231051:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574074.231065:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5986 00000100:00080000:1.0:1788574074.231078:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574074.231086:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574074.231132:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d180 x1875449616399616/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574074.231168:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574074.231184:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788574074.231187:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574074.231192:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5986 00000100:00080000:1.0:1788574074.231197:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574074.231202:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574074.231215:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574074.231228:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616399616:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574074.231234:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ca80 x1875449616399744/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574074.231249:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574074.231253:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788574074.231256:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574074.231261:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5986 00000100:00080000:1.0:1788574074.231266:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574074.231270:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574074.231276:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574074.231287:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d500 x1875449616399872/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574074.231321:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574074.231373:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574074/real 1788574074] req@ffff9ada4566d180 x1875449616399616/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574099 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574074.231398:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d180 x1875449616399616/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574099 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574074.231407:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574074.231411:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574074.231414:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616399616:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574074.231439:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616399744:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574074.231464:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574074.231479:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574074/real 1788574074] req@ffff9ada4566ca80 x1875449616399744/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574099 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574074.231493:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ca80 x1875449616399744/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574099 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574074.231503:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574074.231507:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574074.231510:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616399744:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574074.231515:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616399872:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574074.231537:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574074.231549:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574074/real 1788574074] req@ffff9ada4566d500 x1875449616399872/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574099 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574074.231570:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d500 x1875449616399872/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574099 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574074.231580:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574074.231582:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574074.231585:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616399872:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574074.422997:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574074.743025:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574075.446962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574075.767028:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574076.470958:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574076.791009:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574077.494957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574077.815059:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574078.518961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574078.838944:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574079.350977:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574079.350989:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574079.350993:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574079.351003:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5991 00000100:00080000:2.0:1788574079.351014:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574079.351023:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574079.351056:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616400000/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574079.351089:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574079.351092:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574079.351093:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574079.351096:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5991 00000100:00080000:2.0:1788574079.351099:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574079.351101:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574079.351110:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574079.351119:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616400128/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574079.351127:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574079.351131:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1788574079.351131:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616400000:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788574079.351132:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574079.351158:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5991 00000100:00080000:2.0:1788574079.351161:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574079.351164:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574079.351168:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574079.351178:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616400256/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574079.351204:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574079.351241:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574079/real 1788574079] req@ffff9ada60625880 x1875449616400000/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574104 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574079.351263:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616400000/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574104 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574079.351271:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574079.351276:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574079.351280:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616400000:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574079.351304:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616400128:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574079.351329:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574079.351339:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574079/real 1788574079] req@ffff9ada60625500 x1875449616400128/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574104 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574079.351348:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616400128/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574104 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574079.351358:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574079.351360:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574079.351362:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616400128:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574079.351366:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616400256:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574079.351378:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574079.351384:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574079/real 1788574079] req@ffff9ada60625c00 x1875449616400256/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574104 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574079.351391:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616400256/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574104 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574079.351396:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574079.351398:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574079.351400:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616400256:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574079.542949:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574079.862989:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574080.566939:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574080.887031:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574081.590939:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574081.910984:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574082.614943:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574082.935096:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574083.638938:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574083.959013:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788574084.470976:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574084.470985:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788574084.470988:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574084.470997:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 5997 00000100:00080000:1.0:1788574084.471006:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574084.471012:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574084.471046:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f480 x1875449616400384/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574084.471069:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574084.471072:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788574084.471074:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574084.471077:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5997 00000100:00080000:1.0:1788574084.471079:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574084.471081:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574084.471088:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574084.471118:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e300 x1875449616400512/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574084.471129:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574084.471132:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788574084.471134:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574084.471138:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 5997 00000100:00080000:1.0:1788574084.471143:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574084.471146:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574084.471151:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574084.471163:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ea00 x1875449616400640/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574084.471169:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616400384:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788574084.471236:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574084.471266:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574084/real 1788574084] req@ffff9ada4566f480 x1875449616400384/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574109 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574084.471281:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f480 x1875449616400384/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574109 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574084.471289:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574084.471292:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574084.471295:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616400384:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574084.471300:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616400512:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574084.471317:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574084.471324:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574084/real 1788574084] req@ffff9ada4566e300 x1875449616400512/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574109 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574084.471332:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e300 x1875449616400512/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574109 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574084.471337:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574084.471339:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574084.471341:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616400512:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574084.471359:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616400640:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574084.471369:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574084.471374:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574084/real 1788574084] req@ffff9ada4566ea00 x1875449616400640/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574109 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574084.471382:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ea00 x1875449616400640/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574109 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574084.471387:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574084.471388:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574084.471390:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616400640:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574084.663010:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574084.982993:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574085.686954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574086.006955:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574086.710922:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574087.031036:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574087.734963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574088.055023:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574088.758948:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574089.079034:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788574089.591008:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574089.591016:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788574089.591020:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574089.591029:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6002 00000100:00080000:1.0:1788574089.591038:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574089.591044:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574089.591096:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f800 x1875449616400768/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574089.591124:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574089.591127:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788574089.591128:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574089.591131:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6002 00000100:00080000:1.0:1788574089.591134:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574089.591136:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574089.591144:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574089.591155:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566dc00 x1875449616400896/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574089.591163:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574089.591166:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788574089.591168:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1788574089.591171:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616400768:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788574089.591173:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6002 00000100:00080000:1.0:1788574089.591180:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574089.591182:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574089.591185:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574089.591193:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ed80 x1875449616401024/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574089.591228:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574089.591258:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574089/real 1788574089] req@ffff9ada4566f800 x1875449616400768/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574114 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574089.591272:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f800 x1875449616400768/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574114 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574089.591280:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574089.591283:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574089.591286:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616400768:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574089.591304:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616400896:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574089.591318:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574089.591326:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574089/real 1788574089] req@ffff9ada4566dc00 x1875449616400896/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574114 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574089.591335:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566dc00 x1875449616400896/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574114 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574089.591340:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574089.591351:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574089.591354:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616400896:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574089.591360:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616401024:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574089.591373:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574089.591379:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574089/real 1788574089] req@ffff9ada4566ed80 x1875449616401024/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574114 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574089.591392:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ed80 x1875449616401024/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574114 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574089.591398:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574089.591400:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574089.591402:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616401024:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574089.782960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574090.103077:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574090.806969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574091.127031:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574091.830925:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574092.151025:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574092.854930:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574093.175022:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574093.878954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574094.198994:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574094.710968:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574094.710976:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574094.710979:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574094.710988:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6007 00000100:00080000:2.0:1788574094.710998:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574094.711004:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574094.711049:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616401152/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574094.711075:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574094.711079:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574094.711081:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574094.711085:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6007 00000100:00080000:2.0:1788574094.711089:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574094.711093:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574094.711104:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574094.711120:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616401152:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788574094.711121:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616401280/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574094.711140:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574094.711143:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574094.711146:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574094.711159:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6007 00000100:00080000:2.0:1788574094.711179:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574094.711185:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000800:00000100:3.0:1788574094.711190:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000008:00100000:2.0:1788574094.711195:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574094.711219:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616401408/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574094.711236:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574094/real 1788574094] req@ffff9ada60625f80 x1875449616401152/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574119 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574094.711259:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616401152/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574119 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574094.711272:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574094.711276:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574094.711281:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616401152:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574094.711304:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616401280:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574094.711328:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574094.711341:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574094/real 1788574094] req@ffff9ada60624e00 x1875449616401280/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574119 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574094.711355:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616401280/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574119 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574094.711372:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574094.711376:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574094.711380:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616401280:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574094.711387:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616401408:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574094.711407:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574094.711419:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574094/real 1788574094] req@ffff9ada60627480 x1875449616401408/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574119 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574094.711432:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616401408/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574119 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574094.711442:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574094.711445:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574094.711448:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616401408:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574094.902948:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574095.223031:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574095.926974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574096.246970:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574096.950930:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574097.271181:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574097.974980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574098.294979:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574098.998944:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574099.319047:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574099.830940:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574099.830948:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574099.830952:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574099.830959:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6012 00000100:00080000:0.0:1788574099.830968:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574099.830974:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574099.831031:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616401536/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574099.831057:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574099.831061:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574099.831063:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574099.831067:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6012 00000100:00080000:0.0:1788574099.831072:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574099.831076:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574099.831087:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574099.831103:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616401536:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574099.831104:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616401664/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574099.831121:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574099.831124:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574099.831125:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574099.831129:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6012 00000100:00080000:0.0:1788574099.831133:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574099.831136:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574099.831141:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574099.831151:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616401792/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574099.831174:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574099.831205:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574099/real 1788574099] req@ffff9ada4557c700 x1875449616401536/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574124 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574099.831219:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616401536/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574124 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574099.831227:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574099.831231:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574099.831234:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616401536:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574099.831253:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616401664:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574099.831268:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574099.831275:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574099/real 1788574099] req@ffff9ada4557f480 x1875449616401664/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574124 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574099.831283:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616401664/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574124 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574099.831288:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574099.831290:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574099.831292:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616401664:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574099.831296:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616401792:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574099.831307:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574099.831313:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574099/real 1788574099] req@ffff9ada4557ca80 x1875449616401792/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574124 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574099.831320:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616401792/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574124 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574099.831325:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574099.831327:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574099.831329:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616401792:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574100.022952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574100.343013:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574101.046940:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574101.367071:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574102.070999:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574102.391031:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574103.094977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574103.415011:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574104.119020:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574104.439162:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574104.951018:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574104.951034:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574104.951039:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574104.951050:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6017 00000100:00080000:0.0:1788574104.951062:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574104.951070:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574104.951117:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616401920/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574104.951147:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574104.951150:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574104.951151:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574104.951154:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6017 00000100:00080000:0.0:1788574104.951157:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574104.951159:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574104.951169:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574104.951180:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616402048/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574104.951189:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574104.951191:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574104.951193:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574104.951197:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6017 00000100:00080000:0.0:1788574104.951199:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574104.951201:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574104.951204:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574104.951213:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616402176/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574104.951216:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616401920:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788574104.951293:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574104.951325:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574104/real 1788574104] req@ffff9ada4557f800 x1875449616401920/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574129 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574104.951345:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616401920/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574129 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574104.951354:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574104.951357:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574104.951360:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616401920:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574104.951365:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616402048:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574104.951380:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574104.951388:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574104/real 1788574104] req@ffff9ada4557d880 x1875449616402048/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574129 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574104.951396:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616402048/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574129 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574104.951401:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574104.951403:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574104.951405:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616402048:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574104.951425:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616402176:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574104.951437:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574104.951444:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574104/real 1788574104] req@ffff9ada4557dc00 x1875449616402176/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574129 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574104.951456:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616402176/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574129 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574104.951461:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574104.951463:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574104.951465:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616402176:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574105.142973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574105.463019:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574106.166938:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574106.486993:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574107.190955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574107.511033:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574108.214956:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574108.535033:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574109.238970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574109.558995:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574110.070950:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574110.070958:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574110.070961:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574110.070970:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6022 00000100:00080000:0.0:1788574110.070979:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574110.070985:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574110.071020:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616402304/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574110.071043:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574110.071046:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574110.071047:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574110.071050:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6022 00000100:00080000:0.0:1788574110.071053:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574110.071056:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574110.071063:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574110.071072:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616402432/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574110.071079:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574110.071082:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574110.071083:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574110.071086:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6022 00000100:00080000:0.0:1788574110.071087:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574110.071090:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574110.071094:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574110.071096:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616402304:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574110.071102:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616402560/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574110.071158:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574110.071202:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574110/real 1788574110] req@ffff9ada4557e300 x1875449616402304/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574135 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574110.071223:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616402304/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574135 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574110.071247:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574110.071252:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574110.071255:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616402304:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574110.071261:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616402432:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574110.071281:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574110.071290:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574110/real 1788574110] req@ffff9ada4557fb80 x1875449616402432/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574135 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574110.071299:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616402432/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574135 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574110.071305:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574110.071308:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574110.071310:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616402432:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574110.071327:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616402560:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574110.071339:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574110.071345:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574110/real 1788574110] req@ffff9ada4557ed80 x1875449616402560/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574135 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574110.071352:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616402560/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574135 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574110.071358:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574110.071359:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574110.071361:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616402560:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574110.262917:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574110.583044:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574111.286952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574111.607015:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574112.310945:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574112.631002:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574113.335105:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574113.655011:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574114.358949:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574114.679027:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788574115.191024:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574115.191032:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788574115.191035:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574115.191044:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6027 00000100:00080000:3.0:1788574115.191053:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574115.191059:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574115.191094:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2a00 x1875449616402688/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574115.191121:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574115.191124:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788574115.191125:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574115.191127:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6027 00000100:00080000:3.0:1788574115.191130:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574115.191132:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574115.191139:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574115.191148:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0e00 x1875449616402816/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574115.191156:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574115.191158:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788574115.191162:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1788574115.191163:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616402688:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788574115.191166:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6027 00000100:00080000:3.0:1788574115.191172:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574115.191175:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574115.191178:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574115.191187:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1500 x1875449616402944/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788574115.191269:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616402816:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574115.191305:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616402944:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574115.382953:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574115.702960:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574116.406933:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574116.726982:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574117.430978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574117.751002:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0:1788574118.263133:0:2371:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:3.0:1788574118.263186:0:2371:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1788574118.263202:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:3.0:1788574118.263242:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:3.0:1788574118.263253:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:1.0:1788574118.263293:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574115/real 1788574118] req@ffff9ada604e0e00 x1875449616402816/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574140 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574118.263334:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0e00 x1875449616402816/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574140 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574118.263349:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574118.263354:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574118.263359:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616402816:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574118.263369:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574115/real 1788574118] req@ffff9ada604e1500 x1875449616402944/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574140 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574118.263384:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1500 x1875449616402944/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574140 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574118.263393:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574118.263397:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574118.263401:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616402944:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574118.263408:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574115/real 1788574118] req@ffff9ada604e2a00 x1875449616402688/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574140 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574118.263430:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2a00 x1875449616402688/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574140 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574118.263439:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574118.263443:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574118.263446:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616402688:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574118.454955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574118.774971:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574119.478963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574119.799063:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788574120.311024:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574120.311039:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788574120.311045:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574120.311058:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6032 00000100:00080000:1.0:1788574120.311071:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574120.311080:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574120.311121:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c000 x1875449616403072/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574120.311154:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574120.311157:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788574120.311158:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574120.311162:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6032 00000100:00080000:1.0:1788574120.311164:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574120.311167:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574120.311179:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574120.311190:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566df80 x1875449616403200/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574120.311196:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616403072:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788574120.311206:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574120.311218:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788574120.311220:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574120.311226:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6032 00000100:00080000:1.0:1788574120.311230:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574120.311234:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574120.311240:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574120.311256:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566fb80 x1875449616403328/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574120.311300:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574120.311345:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574120/real 1788574120] req@ffff9ada4566c000 x1875449616403072/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574145 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574120.311383:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c000 x1875449616403072/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574145 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574120.311396:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574120.311402:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574120.311407:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616403072:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574120.311480:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616403200:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574120.311512:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574120.311527:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574120/real 1788574120] req@ffff9ada4566df80 x1875449616403200/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574145 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574120.311542:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566df80 x1875449616403200/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574145 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574120.311553:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574120.311557:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574120.311562:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616403200:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574120.311569:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616403328:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574120.311593:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574120.311607:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574120/real 1788574120] req@ffff9ada4566fb80 x1875449616403328/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574145 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574120.311620:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566fb80 x1875449616403328/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574145 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574120.311630:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574120.311634:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574120.311637:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616403328:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574120.502951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574120.823052:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574121.526954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574121.846954:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574122.550974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574122.871024:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574123.574913:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574123.895047:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574124.598951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574124.918978:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574125.431011:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574125.431022:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574125.431027:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574125.431038:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6037 00000100:00080000:2.0:1788574125.431050:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574125.431058:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574125.431103:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616403456/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574125.431132:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574125.431135:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574125.431137:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574125.431139:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6037 00000100:00080000:2.0:1788574125.431142:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574125.431144:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574125.431155:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574125.431165:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616403584/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574125.431173:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574125.431175:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574125.431177:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574125.431180:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6037 00000100:00080000:2.0:1788574125.431182:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574125.431184:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574125.431188:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574125.431198:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616403712/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574125.431220:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616403456:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788574125.431329:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574125.431371:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574125/real 1788574125] req@ffff9ada60625180 x1875449616403456/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574150 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574125.431392:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616403456/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574150 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574125.431405:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574125.431410:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574125.431415:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616403456:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574125.431423:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616403584:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574125.431449:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574125.431462:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574125/real 1788574125] req@ffff9ada60626d80 x1875449616403584/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574150 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574125.431476:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616403584/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574150 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574125.431485:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574125.431489:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574125.431492:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616403584:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574125.431519:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616403712:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574125.431544:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574125.431556:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574125/real 1788574125] req@ffff9ada60627b80 x1875449616403712/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574150 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574125.431568:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616403712/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574150 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574125.431577:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574125.431580:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574125.431584:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616403712:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574125.622933:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574125.943053:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574126.646962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574126.967038:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574127.670926:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574127.990978:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574128.694945:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574129.015016:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574129.718916:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574130.039011:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574130.551009:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574130.551021:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574130.551025:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574130.551037:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6043 00000100:00080000:2.0:1788574130.551047:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574130.551056:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574130.551099:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626300 x1875449616403840/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574130.551141:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574130.551146:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574130.551149:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574130.551154:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6043 00000100:00080000:2.0:1788574130.551158:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574130.551162:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574130.551172:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574130.551188:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627800 x1875449616403968/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574130.551195:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616403840:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788574130.551208:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574130.551213:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574130.551215:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574130.551220:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6043 00000100:00080000:2.0:1788574130.551225:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574130.551230:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574130.551236:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574130.551251:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616404096/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574130.551261:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574130.551302:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574130/real 1788574130] req@ffff9ada60626300 x1875449616403840/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574155 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574130.551320:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626300 x1875449616403840/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574155 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574130.551327:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574130.551330:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574130.551333:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616403840:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574130.551352:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616403968:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574130.551369:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574130.551378:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574130/real 1788574130] req@ffff9ada60627800 x1875449616403968/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574155 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574130.551385:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627800 x1875449616403968/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574155 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574130.551391:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574130.551393:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574130.551395:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616403968:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574130.551401:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616404096:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574130.551411:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574130.551417:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574130/real 1788574130] req@ffff9ada60627b80 x1875449616404096/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574155 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574130.551425:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616404096/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574155 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574130.551430:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574130.551431:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574130.551433:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616404096:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574130.742968:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574131.062995:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574131.766968:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574132.086995:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574132.790951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574133.111147:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574133.814967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574134.135062:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574134.838953:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574135.159031:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574135.671005:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574135.671013:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574135.671016:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574135.671025:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6048 00000100:00080000:0.0:1788574135.671034:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574135.671039:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574135.671074:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616404224/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574135.671097:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574135.671100:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574135.671101:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574135.671103:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6048 00000100:00080000:0.0:1788574135.671106:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574135.671108:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574135.671115:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574135.671122:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616404352/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574135.671129:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574135.671131:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574135.671132:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574135.671134:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6048 00000100:00080000:0.0:1788574135.671136:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574135.671138:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574135.671140:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574135.671146:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d500 x1875449616404480/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574135.671170:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616404224:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788574135.671225:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574135.671254:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574135/real 1788574135] req@ffff9ada4557ea00 x1875449616404224/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574160 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574135.671269:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616404224/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574160 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574135.671277:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574135.671281:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574135.671285:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616404224:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574135.671294:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616404352:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574135.671318:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574135.671332:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574135/real 1788574135] req@ffff9ada4557e680 x1875449616404352/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574160 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574135.671347:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616404352/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574160 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574135.671354:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574135.671356:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574135.671359:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616404352:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574135.671363:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616404480:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574135.671379:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574135.671387:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574135/real 1788574135] req@ffff9ada4557d500 x1875449616404480/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574160 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574135.671395:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d500 x1875449616404480/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574160 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574135.671400:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574135.671402:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574135.671405:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616404480:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574135.862949:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574136.183046:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574136.886963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574137.206957:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574137.910962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574138.231021:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574138.934970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574139.255036:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574139.958954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574140.279069:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574140.790944:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574140.790954:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574140.790957:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574140.790966:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6053 00000100:00080000:0.0:1788574140.790975:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574140.790981:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574140.791012:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557df80 x1875449616404608/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574140.791034:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574140.791036:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574140.791038:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574140.791040:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6053 00000100:00080000:0.0:1788574140.791042:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574140.791044:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574140.791052:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574140.791061:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d180 x1875449616404736/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574140.791068:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574140.791071:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574140.791073:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574140.791075:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6053 00000100:00080000:0.0:1788574140.791077:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574140.791081:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574140.791085:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616404608:192.168.202.113@tcp:250:kworker.0 00000008:00100000:0.0:1788574140.791086:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574140.791098:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d500 x1875449616404864/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574140.791149:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574140.791180:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574140/real 1788574140] req@ffff9ada4557df80 x1875449616404608/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574165 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574140.791198:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557df80 x1875449616404608/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574165 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574140.791210:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574140.791214:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574140.791218:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616404608:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574140.791225:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616404736:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574140.791243:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574140.791253:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574140/real 1788574140] req@ffff9ada4557d180 x1875449616404736/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574165 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574140.791266:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d180 x1875449616404736/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574165 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574140.791274:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574140.791278:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574140.791281:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616404736:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574140.791307:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616404864:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574140.791322:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574140.791331:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574140/real 1788574140] req@ffff9ada4557d500 x1875449616404864/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574165 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574140.791344:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d500 x1875449616404864/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574165 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574140.791362:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574140.791365:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574140.791369:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616404864:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574140.982935:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574141.303055:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574142.006933:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574142.327012:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574143.030945:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574143.350987:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574144.054967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574144.375004:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574145.078927:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574145.399012:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574145.910986:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574145.910993:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574145.910997:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574145.911006:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6058 00000100:00080000:2.0:1788574145.911015:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574145.911021:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574145.911054:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616404992/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574145.911078:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574145.911081:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574145.911082:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574145.911085:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6058 00000100:00080000:2.0:1788574145.911087:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574145.911090:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574145.911097:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574145.911106:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616405120/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574145.911115:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574145.911118:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574145.911120:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1788574145.911122:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616404992:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788574145.911124:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6058 00000100:00080000:2.0:1788574145.911129:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574145.911135:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574145.911141:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574145.911154:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616405248/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574145.911193:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574145.911229:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574145/real 1788574145] req@ffff9ada60626d80 x1875449616404992/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574170 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574145.911245:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616404992/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574170 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574145.911253:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574145.911256:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574145.911259:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616404992:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574145.911278:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616405120:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574145.911295:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574145.911303:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574145/real 1788574145] req@ffff9ada60625180 x1875449616405120/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574170 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574145.911312:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616405120/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574170 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574145.911318:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574145.911320:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574145.911323:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616405120:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574145.911327:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616405248:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574145.911337:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574145.911342:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574145/real 1788574145] req@ffff9ada60627480 x1875449616405248/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574170 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574145.911350:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616405248/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574170 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574145.911355:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574145.911357:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574145.911359:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616405248:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574146.103017:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574146.422966:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574147.126931:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574147.447006:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574148.150957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574148.471039:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574149.174954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574149.495051:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574150.198965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574150.519058:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574151.030964:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574151.030972:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574151.030975:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574151.030983:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6063 00000100:00080000:2.0:1788574151.030992:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574151.030997:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574151.031031:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616405376/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574151.031061:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574151.031065:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574151.031073:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574151.031077:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6063 00000100:00080000:2.0:1788574151.031080:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574151.031084:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574151.031092:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574151.031101:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616405504/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574151.031110:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574151.031113:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574151.031114:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574151.031118:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6063 00000100:00080000:2.0:1788574151.031122:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574151.031129:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574151.031130:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616405376:192.168.202.113@tcp:250:kworker.0 00000008:00100000:2.0:1788574151.031134:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574151.031144:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616405632/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574151.031226:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574151.031275:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574151/real 1788574151] req@ffff9ada60624e00 x1875449616405376/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574176 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574151.031300:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616405376/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574176 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574151.031314:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574151.031319:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574151.031324:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616405376:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574151.031357:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616405504:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574151.031384:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574151.031399:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574151/real 1788574151] req@ffff9ada60625f80 x1875449616405504/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574176 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574151.031413:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616405504/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574176 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574151.031424:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574151.031427:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574151.031431:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616405504:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574151.031439:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616405632:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574151.031460:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574151.031471:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574151/real 1788574151] req@ffff9ada60625c00 x1875449616405632/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574176 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574151.031485:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616405632/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574176 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574151.031508:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574151.031513:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574151.031517:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616405632:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574151.223073:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574151.543077:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574152.246968:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574152.566962:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574153.270914:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574153.591083:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574154.294931:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574154.615272:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574155.318931:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574155.639001:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574156.150998:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574156.151007:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574156.151011:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574156.151019:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6068 00000100:00080000:0.0:1788574156.151027:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574156.151033:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574156.151066:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616405760/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574156.151089:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574156.151091:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574156.151093:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574156.151096:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6068 00000100:00080000:0.0:1788574156.151098:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574156.151100:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574156.151107:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574156.151118:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616405888/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574156.151130:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574156.151134:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574156.151137:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1788574156.151143:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616405760:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788574156.151144:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6068 00000100:00080000:0.0:1788574156.151151:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574156.151158:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574156.151163:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574156.151184:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616406016/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574156.151234:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574156.151271:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574156/real 1788574156] req@ffff9ada4557e680 x1875449616405760/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574181 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574156.151291:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616405760/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574181 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574156.151302:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574156.151306:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574156.151317:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616405760:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574156.151325:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616405888:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574156.151365:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574156.151381:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574156/real 1788574156] req@ffff9ada4557ea00 x1875449616405888/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574181 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574156.151394:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616405888/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574181 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574156.151402:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574156.151406:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574156.151409:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616405888:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574156.151433:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616406016:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574156.151452:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574156.151463:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574156/real 1788574156] req@ffff9ada4557ed80 x1875449616406016/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574181 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574156.151475:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616406016/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574181 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574156.151483:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574156.151487:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574156.151490:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616406016:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574156.342959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574156.663018:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574157.366915:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574157.687035:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574158.390957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574158.710976:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574159.414934:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574159.734981:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574160.438928:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574160.759080:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574161.270939:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574161.270947:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574161.270950:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574161.270958:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6073 00000100:00080000:0.0:1788574161.270967:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574161.270973:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574161.271006:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616406144/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574161.271026:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574161.271029:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574161.271030:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574161.271033:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6073 00000100:00080000:0.0:1788574161.271035:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574161.271037:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574161.271045:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574161.271055:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616406272/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574161.271062:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574161.271065:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574161.271072:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574161.271078:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6073 00000100:00100000:3.0:1788574161.271080:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616406144:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788574161.271081:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574161.271087:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574161.271094:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574161.271103:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616406400/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574161.271161:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574161.271212:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574161/real 1788574161] req@ffff9ada4557fb80 x1875449616406144/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574186 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574161.271243:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616406144/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574186 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574161.271255:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574161.271259:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574161.271262:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616406144:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574161.271271:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616406272:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574161.271296:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574161.271311:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574161/real 1788574161] req@ffff9ada4557e300 x1875449616406272/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574186 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574161.271324:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616406272/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574186 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574161.271332:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574161.271336:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574161.271349:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616406272:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574161.271376:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616406400:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574161.271398:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574161.271410:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574161/real 1788574161] req@ffff9ada4557dc00 x1875449616406400/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574186 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574161.271423:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616406400/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574186 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574161.271432:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574161.271435:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574161.271439:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616406400:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574161.462915:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574161.783034:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574162.486967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574162.807055:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574163.510963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574163.831020:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574164.534944:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574164.854944:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574165.558954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574165.879007:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574166.390943:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574166.390958:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574166.390963:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574166.390977:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6078 00000100:00080000:0.0:1788574166.390991:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574166.391000:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574166.391067:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616406528/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574166.391102:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574166.391107:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574166.391110:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574166.391114:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6078 00000100:00080000:0.0:1788574166.391120:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574166.391127:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574166.391140:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574166.391157:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616406656/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574166.391174:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616406528:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788574166.391176:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574166.391190:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574166.391194:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574166.391200:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6078 00000100:00080000:0.0:1788574166.391205:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574166.391211:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574166.391218:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574166.391236:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616406784/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574166.391278:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574166.391318:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574166/real 1788574166] req@ffff9ada4557d880 x1875449616406528/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574191 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574166.391339:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616406528/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574191 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574166.391352:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574166.391358:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574166.391362:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616406528:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574166.391390:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616406656:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574166.391417:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574166.391432:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574166/real 1788574166] req@ffff9ada4557f800 x1875449616406656/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574191 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574166.391447:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616406656/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574191 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574166.391455:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574166.391459:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574166.391463:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616406656:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574166.391470:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616406784:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574166.391490:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574166.391500:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574166/real 1788574166] req@ffff9ada4557ca80 x1875449616406784/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574191 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574166.391513:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616406784/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574191 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574166.391522:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574166.391525:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574166.391528:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616406784:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574166.582930:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574166.903078:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574167.606964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574167.927065:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574168.630946:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574168.951083:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574169.654984:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574169.975061:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574170.678972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574170.999027:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574171.511036:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574171.511050:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574171.511055:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574171.511069:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6084 00000100:00080000:2.0:1788574171.511084:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574171.511093:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574171.511145:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616406912/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574171.511180:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574171.511185:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574171.511187:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574171.511192:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6084 00000100:00080000:2.0:1788574171.511196:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574171.511201:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574171.511213:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574171.511229:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616407040/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574171.511243:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574171.511247:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574171.511250:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574171.511259:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6084 00000100:00100000:3.0:1788574171.511261:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616406912:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788574171.511264:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574171.511275:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574171.511282:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574171.511297:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616407168/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574171.511346:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574171.511389:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574171/real 1788574171] req@ffff9ada60625500 x1875449616406912/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574196 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574171.511410:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616406912/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574196 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574171.511422:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574171.511427:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574171.511431:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616406912:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574171.511458:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616407040:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574171.511482:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574171.511495:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574171/real 1788574171] req@ffff9ada60625880 x1875449616407040/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574196 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574171.511508:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616407040/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574196 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574171.511518:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574171.511522:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574171.511526:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616407040:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574171.511536:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616407168:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574171.511556:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574171.511568:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574171/real 1788574171] req@ffff9ada60624a80 x1875449616407168/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574196 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574171.511582:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616407168/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574196 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574171.511601:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574171.511605:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574171.511609:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616407168:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574171.702952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574172.023027:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574172.726972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574173.047049:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574173.750974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574174.071009:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574174.774962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574175.095058:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574175.798967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574176.119081:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574176.630998:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574176.631008:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574176.631012:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574176.631030:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6089 00000100:00080000:0.0:1788574176.631043:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574176.631052:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574176.631096:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616407296/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574176.631128:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574176.631131:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574176.631132:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574176.631140:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6089 00000100:00080000:0.0:1788574176.631142:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574176.631145:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574176.631155:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574176.631165:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616407424/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574176.631172:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574176.631176:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574176.631177:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574176.631182:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6089 00000100:00080000:0.0:1788574176.631185:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574176.631187:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574176.631190:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574176.631196:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c000 x1875449616407552/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574176.631216:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616407296:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788574176.631281:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574176.631316:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574176/real 1788574176] req@ffff9ada4557f480 x1875449616407296/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574201 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574176.631330:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616407296/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574201 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574176.631338:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574176.631342:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574176.631346:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616407296:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574176.631351:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616407424:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574176.631365:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574176.631372:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574176/real 1788574176] req@ffff9ada4557c700 x1875449616407424/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574201 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574176.631380:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616407424/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574201 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574176.631385:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574176.631387:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574176.631389:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616407424:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574176.631408:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616407552:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574176.631419:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574176.631425:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574176/real 1788574176] req@ffff9ada4557c000 x1875449616407552/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574201 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574176.631432:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c000 x1875449616407552/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574201 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574176.631437:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574176.631439:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574176.631441:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616407552:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574176.822957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574177.143010:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574177.846942:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574178.166989:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574178.870930:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574179.191046:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574179.895071:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574180.215001:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574180.918915:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574181.238968:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574181.750965:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574181.750974:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574181.750978:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574181.750986:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6094 00000100:00080000:0.0:1788574181.750996:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574181.751002:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574181.751035:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ce00 x1875449616407680/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574181.751061:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574181.751064:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574181.751065:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574181.751072:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6094 00000100:00080000:0.0:1788574181.751074:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574181.751077:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574181.751085:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574181.751100:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c380 x1875449616407808/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574181.751111:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574181.751115:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574181.751118:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1788574181.751120:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616407680:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788574181.751125:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6094 00000100:00080000:0.0:1788574181.751130:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574181.751135:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574181.751140:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574181.751156:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c000 x1875449616407936/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574181.751223:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616407808:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574181.751246:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616407936:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574181.942932:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574182.263035:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574182.966964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574183.287173:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574183.990977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574184.311038:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1788574184.823120:0:2370:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:2.0:1788574184.823164:0:2370:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1788574184.823181:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788574184.823225:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788574184.823235:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:3.0:1788574184.823275:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574181/real 1788574184] req@ffff9ada4557c380 x1875449616407808/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574206 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574184.823305:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c380 x1875449616407808/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574206 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574184.823318:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574184.823324:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574184.823340:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616407808:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574184.823350:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574181/real 1788574184] req@ffff9ada4557c000 x1875449616407936/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574206 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574184.823364:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c000 x1875449616407936/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574206 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574184.823374:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574184.823378:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574184.823382:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616407936:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574184.823389:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574181/real 1788574184] req@ffff9ada4557ce00 x1875449616407680/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574206 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574184.823403:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ce00 x1875449616407680/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574206 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574184.823410:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574184.823412:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574184.823415:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616407680:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574185.014946:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574185.335036:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574186.038956:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574186.358980:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788574186.871016:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574186.871028:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788574186.871032:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574186.871044:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6099 00000100:00080000:3.0:1788574186.871056:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574186.871070:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574186.871115:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2680 x1875449616408064/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574186.871150:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574186.871155:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788574186.871158:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574186.871162:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6099 00000100:00080000:3.0:1788574186.871167:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574186.871171:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574186.871183:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574186.871199:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1f80 x1875449616408192/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788574186.871210:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616408064:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788574186.871214:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574186.871220:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788574186.871222:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574186.871227:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6099 00000100:00080000:3.0:1788574186.871232:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574186.871236:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574186.871247:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574186.871262:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3100 x1875449616408320/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788574186.871308:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574186.871360:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574186/real 1788574186] req@ffff9ada604e2680 x1875449616408064/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574211 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574186.871386:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2680 x1875449616408064/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574211 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574186.871399:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574186.871403:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574186.871406:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616408064:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574186.871440:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616408192:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574186.871470:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574186.871485:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574186/real 1788574186] req@ffff9ada604e1f80 x1875449616408192/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574211 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574186.871500:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1f80 x1875449616408192/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574211 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574186.871530:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574186.871534:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574186.871538:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616408192:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574186.871546:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616408320:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574186.871570:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574186.871583:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574186/real 1788574186] req@ffff9ada604e3100 x1875449616408320/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574211 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574186.871592:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3100 x1875449616408320/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574211 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574186.871600:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574186.871605:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574186.871608:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616408320:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574187.062989:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574187.383006:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574188.086925:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574188.407069:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574189.110958:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574189.430964:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574190.134956:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574190.455342:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574191.158968:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574191.479012:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574191.990970:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574191.990978:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574191.990982:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574191.990990:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6104 00000100:00080000:2.0:1788574191.991000:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574191.991006:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574191.991039:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616408448/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574191.991062:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574191.991065:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574191.991066:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574191.991075:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6104 00000100:00080000:2.0:1788574191.991077:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574191.991081:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574191.991089:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574191.991098:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616408576/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574191.991106:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1788574191.991108:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616408448:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788574191.991109:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574191.991113:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574191.991121:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6104 00000100:00080000:2.0:1788574191.991123:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574191.991126:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574191.991129:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574191.991138:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616408704/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788574191.991176:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574191.991207:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574191/real 1788574191] req@ffff9ada60626a00 x1875449616408448/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574216 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574191.991220:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616408448/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574216 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574191.991227:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574191.991230:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574191.991233:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616408448:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574191.991251:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616408576:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574191.991266:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574191.991274:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574191/real 1788574191] req@ffff9ada60624380 x1875449616408576/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574216 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574191.991288:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616408576/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574216 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574191.991294:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574191.991296:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574191.991299:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616408576:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574191.991302:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616408704:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574191.991314:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574191.991321:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574191/real 1788574191] req@ffff9ada60626680 x1875449616408704/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574216 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574191.991328:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616408704/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574216 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574191.991333:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574191.991335:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574191.991336:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616408704:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574192.182929:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574192.503002:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574193.206955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574193.526988:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574194.230937:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574194.551060:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574195.254966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574195.574964:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574196.278961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574196.598995:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574197.110974:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574197.110983:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574197.110987:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574197.110995:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6109 00000100:00080000:2.0:1788574197.111004:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574197.111010:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574197.111043:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627100 x1875449616408832/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574197.111068:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574197.111070:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574197.111072:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574197.111075:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6109 00000100:00080000:2.0:1788574197.111077:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574197.111079:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574197.111086:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574197.111094:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624700 x1875449616408960/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574197.111102:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574197.111105:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574197.111107:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574197.111109:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6109 00000100:00080000:2.0:1788574197.111111:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574197.111114:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574197.111120:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574197.111124:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616408832:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788574197.111136:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616409088/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788574197.111182:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574197.111212:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574197/real 1788574197] req@ffff9ada60627100 x1875449616408832/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574222 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574197.111226:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627100 x1875449616408832/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574222 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574197.111238:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574197.111241:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574197.111244:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616408832:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574197.111249:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616408960:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574197.111263:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574197.111270:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574197/real 1788574197] req@ffff9ada60624700 x1875449616408960/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574222 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574197.111301:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624700 x1875449616408960/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574222 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574197.111308:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574197.111311:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574197.111313:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616408960:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574197.111333:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616409088:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574197.111348:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574197.111355:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574197/real 1788574197] req@ffff9ada60626680 x1875449616409088/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574222 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574197.111363:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616409088/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574222 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574197.111368:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574197.111370:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574197.111372:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616409088:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574197.302976:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574197.623220:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574198.326962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574198.646971:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574199.350951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574199.671042:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574200.374925:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574200.695059:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574201.398950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574201.718986:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574202.230921:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574202.230929:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574202.230932:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574202.230941:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6114 00000100:00080000:2.0:1788574202.230950:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574202.230955:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574202.231001:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616409216/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574202.231042:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574202.231045:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574202.231046:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574202.231049:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6114 00000100:00080000:2.0:1788574202.231052:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574202.231055:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574202.231063:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574202.231074:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616409344/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574202.231082:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574202.231086:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574202.231087:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574202.231091:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6114 00000100:00080000:2.0:1788574202.231093:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574202.231096:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574202.231099:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574202.231106:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616409216:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788574202.231110:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616409472/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788574202.231182:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574202.231239:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574202/real 1788574202] req@ffff9ada60624380 x1875449616409216/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574227 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574202.231264:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616409216/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574227 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574202.231278:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574202.231283:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574202.231289:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616409216:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574202.231298:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616409344:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574202.231342:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574202.231352:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574202/real 1788574202] req@ffff9ada60626a00 x1875449616409344/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574227 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574202.231360:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616409344/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574227 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574202.231366:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574202.231368:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574202.231371:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616409344:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574202.231389:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616409472:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574202.231401:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574202.231407:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574202/real 1788574202] req@ffff9ada60624a80 x1875449616409472/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574227 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574202.231415:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616409472/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574227 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574202.231420:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574202.231421:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574202.231423:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616409472:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574202.423000:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574202.743028:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574203.446957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574203.767030:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574204.470994:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574204.791132:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574205.494926:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574205.815021:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574206.518969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574206.839057:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574207.350968:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574207.350976:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574207.350979:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574207.350988:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6119 00000100:00080000:0.0:1788574207.350997:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574207.351003:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574207.351041:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616409600/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574207.351070:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574207.351073:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574207.351074:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574207.351078:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6119 00000100:00080000:0.0:1788574207.351081:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574207.351084:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574207.351091:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574207.351103:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616409728/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574207.351111:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574207.351114:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574207.351116:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574207.351120:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6119 00000100:00080000:0.0:1788574207.351122:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574207.351125:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574207.351128:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574207.351135:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616409600:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574207.351136:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616409856/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788574207.351193:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574207.351221:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574207/real 1788574207] req@ffff9ada4557c700 x1875449616409600/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574232 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574207.351235:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616409600/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574232 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574207.351244:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574207.351246:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574207.351249:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616409600:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574207.351254:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616409728:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574207.351268:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574207.351290:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574207/real 1788574207] req@ffff9ada4557f480 x1875449616409728/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574232 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574207.351304:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616409728/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574232 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574207.351320:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574207.351324:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574207.351327:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616409728:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574207.351352:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616409856:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574207.351369:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574207.351377:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574207/real 1788574207] req@ffff9ada4557ca80 x1875449616409856/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574232 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574207.351385:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616409856/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574232 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574207.351391:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574207.351393:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574207.351395:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616409856:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574207.542923:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574207.862936:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574208.566969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574208.887022:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574209.590919:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574209.911025:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574210.614986:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574210.934969:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574211.638950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574211.959064:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574212.470992:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574212.471001:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574212.471004:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574212.471013:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6125 00000100:00080000:2.0:1788574212.471022:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574212.471029:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574212.471062:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616409984/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574212.471089:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574212.471092:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574212.471093:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574212.471096:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6125 00000100:00080000:2.0:1788574212.471098:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574212.471100:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574212.471107:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574212.471116:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616410112/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574212.471124:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574212.471126:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574212.471128:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574212.471130:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6125 00000100:00080000:2.0:1788574212.471132:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574212.471134:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574212.471136:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574212.471143:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616410240/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788574212.471172:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616409984:192.168.202.113@tcp:250:kworker.0 00000800:00000100:1.0:1788574212.471251:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574212.471291:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574212/real 1788574212] req@ffff9ada60625880 x1875449616409984/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574237 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574212.471308:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616409984/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574237 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574212.471326:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574212.471331:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574212.471336:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616409984:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574212.471346:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616410112:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574212.471373:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574212.471387:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574212/real 1788574212] req@ffff9ada60625500 x1875449616410112/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574237 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574212.471413:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616410112/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574237 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574212.471419:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574212.471422:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574212.471424:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616410112:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574212.471455:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616410240:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574212.471490:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574212.471503:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574212/real 1788574212] req@ffff9ada60625c00 x1875449616410240/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574237 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574212.471516:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616410240/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574237 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574212.471525:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574212.471528:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574212.471530:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616410240:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574212.662990:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574212.983025:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574213.686957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574214.007010:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574214.710994:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574215.030992:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574215.734957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574216.055118:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574216.758957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574217.078960:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574217.590992:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574217.591004:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574217.591008:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574217.591019:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6130 00000100:00080000:2.0:1788574217.591032:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574217.591040:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574217.591087:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616410368/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574217.591129:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574217.591132:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574217.591133:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574217.591138:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6130 00000100:00080000:2.0:1788574217.591141:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574217.591143:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574217.591154:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574217.591163:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616410496/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574217.591171:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574217.591174:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574217.591175:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574217.591178:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6130 00000100:00080000:2.0:1788574217.591180:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574217.591182:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574217.591185:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574217.591192:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616410624/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788574217.591227:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616410368:192.168.202.113@tcp:250:kworker.0 00000800:00000100:1.0:1788574217.591309:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574217.591352:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574217/real 1788574217] req@ffff9ada60625f80 x1875449616410368/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574242 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574217.591373:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616410368/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574242 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574217.591385:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574217.591389:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574217.591391:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616410368:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574217.591398:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616410496:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574217.591418:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574217.591427:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574217/real 1788574217] req@ffff9ada60624e00 x1875449616410496/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574242 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574217.591436:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616410496/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574242 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574217.591442:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574217.591444:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574217.591446:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616410496:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574217.591450:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616410624:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574217.591461:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574217.591467:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574217/real 1788574217] req@ffff9ada60627480 x1875449616410624/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574242 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574217.591475:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616410624/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574242 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574217.591489:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574217.591492:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574217.591496:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616410624:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574217.783068:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574218.103138:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574218.806957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574219.127067:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574219.830986:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574220.150996:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574220.854970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574221.175082:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574221.878951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574222.199051:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788574222.711013:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574222.711021:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788574222.711025:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574222.711033:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6135 00000100:00080000:3.0:1788574222.711041:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574222.711047:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574222.711081:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3b80 x1875449616410752/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574222.711105:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574222.711108:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788574222.711110:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574222.711112:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6135 00000100:00080000:3.0:1788574222.711115:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574222.711117:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574222.711124:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574222.711134:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1c00 x1875449616410880/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574222.711147:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1788574222.711153:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616410752:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788574222.711153:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788574222.711155:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574222.711164:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6135 00000100:00080000:3.0:1788574222.711166:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574222.711168:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574222.711171:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574222.711181:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0000 x1875449616411008/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788574222.711219:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574222.711255:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574222/real 1788574222] req@ffff9ada604e3b80 x1875449616410752/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574247 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574222.711277:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3b80 x1875449616410752/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574247 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574222.711285:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574222.711288:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574222.711291:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616410752:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574222.711309:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616410880:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574222.711333:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574222.711342:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574222/real 1788574222] req@ffff9ada604e1c00 x1875449616410880/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574247 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574222.711351:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1c00 x1875449616410880/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574247 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574222.711357:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574222.711360:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574222.711362:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616410880:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574222.711366:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616411008:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574222.711377:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574222.711384:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574222/real 1788574222] req@ffff9ada604e0000 x1875449616411008/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574247 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574222.711391:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0000 x1875449616411008/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574247 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574222.711396:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574222.711398:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574222.711400:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616411008:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574222.902950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574223.223059:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574223.926951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574224.247057:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574224.950926:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574225.271046:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574225.974961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574226.294975:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574226.998922:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574227.319063:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574227.830999:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574227.831008:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574227.831011:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574227.831019:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6140 00000100:00080000:0.0:1788574227.831028:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574227.831036:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574227.831075:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616411136/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574227.831104:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574227.831108:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574227.831110:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574227.831114:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6140 00000100:00080000:0.0:1788574227.831119:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574227.831124:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574227.831134:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574227.831148:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616411136:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574227.831150:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616411264/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574227.831171:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574227.831175:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574227.831177:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574227.831181:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6140 00000100:00080000:0.0:1788574227.831194:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574227.831198:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574227.831203:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000800:00000100:1.0:1788574227.831212:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574227.831218:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616411392/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788574227.831247:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574227/real 1788574227] req@ffff9ada4557f800 x1875449616411136/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574252 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574227.831264:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616411136/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574252 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574227.831274:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574227.831278:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574227.831283:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616411136:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574227.831307:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616411264:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574227.831326:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574227.831336:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574227/real 1788574227] req@ffff9ada4557d880 x1875449616411264/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574252 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574227.831345:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616411264/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574252 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574227.831353:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574227.831356:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574227.831360:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616411264:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574227.831366:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616411392:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574227.831387:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574227.831400:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574227/real 1788574227] req@ffff9ada4557dc00 x1875449616411392/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574252 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574227.831411:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616411392/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574252 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574227.831417:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574227.831419:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574227.831422:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616411392:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574228.022983:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574228.342984:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574229.046952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574229.367009:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574230.070968:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574230.390961:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574231.094948:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574231.415092:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574232.118974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574232.438942:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574232.950934:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574232.950942:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574232.950946:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574232.950954:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6145 00000100:00080000:0.0:1788574232.950963:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574232.950968:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574232.951001:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616411520/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574232.951024:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574232.951027:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574232.951029:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574232.951031:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6145 00000100:00080000:0.0:1788574232.951033:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574232.951035:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574232.951044:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574232.951051:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616411648/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574232.951058:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574232.951060:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574232.951061:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574232.951063:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6145 00000100:00080000:0.0:1788574232.951065:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574232.951067:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574232.951069:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574232.951074:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616411776/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788574232.951105:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616411520:192.168.202.113@tcp:250:kworker.0 00000800:00000100:1.0:1788574232.951159:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574232.951190:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574232/real 1788574232] req@ffff9ada4557e300 x1875449616411520/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574257 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574232.951203:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616411520/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574257 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574232.951211:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574232.951231:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574232.951234:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616411520:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574232.951240:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616411648:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574232.951254:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574232.951262:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574232/real 1788574232] req@ffff9ada4557fb80 x1875449616411648/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574257 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574232.951270:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616411648/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574257 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574232.951276:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574232.951278:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574232.951280:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616411648:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574232.951284:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616411776:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574232.951293:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574232.951298:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574232/real 1788574232] req@ffff9ada4557ed80 x1875449616411776/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574257 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574232.951305:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616411776/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574257 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574232.951315:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574232.951317:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574232.951318:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616411776:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574233.142965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574233.463023:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574234.166981:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574234.487037:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574235.190942:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574235.510979:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574236.214961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574236.535064:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574237.238946:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574237.559020:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788574238.071003:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574238.071015:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788574238.071020:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574238.071031:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6150 00000100:00080000:3.0:1788574238.071043:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574238.071050:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574238.071095:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2300 x1875449616411904/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574238.071126:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574238.071129:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788574238.071146:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574238.071151:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6150 00000100:00080000:3.0:1788574238.071155:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574238.071160:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574238.071172:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574238.071184:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0a80 x1875449616412032/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788574238.071192:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616411904:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788574238.071195:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574238.071202:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788574238.071204:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574238.071207:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6150 00000100:00080000:3.0:1788574238.071210:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574238.071213:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574238.071218:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574238.071228:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2d80 x1875449616412160/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788574238.071287:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574238.071330:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574238/real 1788574238] req@ffff9ada604e2300 x1875449616411904/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574263 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574238.071347:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2300 x1875449616411904/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574263 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574238.071356:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574238.071359:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574238.071363:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616411904:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574238.071387:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616412032:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574238.071404:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574238.071412:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574238/real 1788574238] req@ffff9ada604e0a80 x1875449616412032/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574263 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574238.071420:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0a80 x1875449616412032/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574263 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574238.071426:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574238.071429:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574238.071431:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616412032:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574238.071435:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616412160:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574238.071446:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574238.071452:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574238/real 1788574238] req@ffff9ada604e2d80 x1875449616412160/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574263 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574238.071460:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2d80 x1875449616412160/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574263 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574238.071465:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574238.071467:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574238.071469:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616412160:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574238.262955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574238.583037:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574239.286937:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574239.607006:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574240.310949:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574240.631038:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574241.334925:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574241.654942:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574242.358910:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574242.679104:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788574243.190965:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574243.190973:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788574243.190977:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574243.190985:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6155 00000100:00080000:1.0:1788574243.190994:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574243.191000:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574243.191033:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566fb80 x1875449616412288/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574243.191076:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574243.191080:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788574243.191082:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574243.191086:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6155 00000100:00080000:1.0:1788574243.191090:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574243.191093:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574243.191102:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616412288:192.168.202.113@tcp:250:kworker.0 00000008:00100000:1.0:1788574243.191103:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574243.191118:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566df80 x1875449616412416/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574243.191126:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574243.191129:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788574243.191130:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574243.191133:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6155 00000100:00080000:1.0:1788574243.191136:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574243.191138:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574243.191142:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574243.191157:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c000 x1875449616412544/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574243.191162:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574243.191199:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574243/real 1788574243] req@ffff9ada4566fb80 x1875449616412288/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574268 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574243.191220:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566fb80 x1875449616412288/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574268 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574243.191232:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574243.191236:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574243.191241:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616412288:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574243.191267:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616412416:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574243.191293:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574243.191307:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574243/real 1788574243] req@ffff9ada4566df80 x1875449616412416/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574268 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574243.191320:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566df80 x1875449616412416/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574268 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574243.191330:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574243.191332:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574243.191335:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616412416:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574243.191339:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616412544:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574243.191354:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574243.191361:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574243/real 1788574243] req@ffff9ada4566c000 x1875449616412544/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574268 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574243.191368:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c000 x1875449616412544/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574268 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574243.191374:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574243.191376:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574243.191378:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616412544:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574243.382971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574243.703011:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574244.406930:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574244.726999:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574245.430957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574245.751042:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574246.454944:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574246.775087:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574247.478968:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574247.798968:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788574248.311050:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574248.311059:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788574248.311063:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574248.311071:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6160 00000100:00080000:3.0:1788574248.311081:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574248.311088:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574248.311124:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1180 x1875449616412672/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574248.311160:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574248.311165:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788574248.311168:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574248.311173:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6160 00000100:00080000:3.0:1788574248.311179:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574248.311185:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574248.311210:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574248.311215:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616412672:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574248.311230:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1880 x1875449616412800/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574248.311244:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574248.311248:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788574248.311251:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574248.311255:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6160 00000100:00080000:3.0:1788574248.311261:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574248.311266:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574248.311271:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574248.311281:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3800 x1875449616412928/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788574248.311308:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616412800:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574248.311359:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616412928:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574248.502914:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574248.823017:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574249.526982:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574249.846985:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574250.550967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574250.871019:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1788574251.383195:0:2372:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:0.0:1788574251.383239:0:2372:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1788574251.383256:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:0.0:1788574251.383298:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:0.0:1788574251.383307:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:1.0:1788574251.383358:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574248/real 1788574251] req@ffff9ada604e1880 x1875449616412800/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574273 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574251.383394:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1880 x1875449616412800/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574273 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574251.383409:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574251.383414:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574251.383419:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616412800:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574251.383428:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574248/real 1788574251] req@ffff9ada604e3800 x1875449616412928/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574273 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574251.383442:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3800 x1875449616412928/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574273 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574251.383451:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574251.383455:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574251.383459:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616412928:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574251.383466:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574248/real 1788574251] req@ffff9ada604e1180 x1875449616412672/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574273 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574251.383480:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1180 x1875449616412672/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574273 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574251.383489:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574251.383492:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574251.383496:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616412672:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574251.574978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574251.895054:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574252.598961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574252.919052:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788574253.430977:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574253.430985:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788574253.430988:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574253.430997:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6165 00000100:00080000:3.0:1788574253.431007:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574253.431013:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574253.431048:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0380 x1875449616413056/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574253.431072:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574253.431074:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788574253.431076:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574253.431078:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6165 00000100:00080000:3.0:1788574253.431081:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574253.431083:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574253.431090:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574253.431100:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0700 x1875449616413184/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574253.431107:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574253.431110:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788574253.431111:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574253.431114:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6165 00000100:00080000:3.0:1788574253.431116:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574253.431120:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574253.431122:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616413056:192.168.202.113@tcp:250:kworker.0 00000008:00100000:3.0:1788574253.431126:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574253.431134:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3480 x1875449616413312/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788574253.431179:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574253.431214:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574253/real 1788574253] req@ffff9ada604e0380 x1875449616413056/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574278 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574253.431234:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0380 x1875449616413056/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574278 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574253.431245:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574253.431249:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574253.431253:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616413056:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574253.431261:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616413184:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574253.431287:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574253.431301:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574253/real 1788574253] req@ffff9ada604e0700 x1875449616413184/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574278 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574253.431315:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0700 x1875449616413184/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574278 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574253.431324:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574253.431327:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574253.431330:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616413184:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574253.431355:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616413312:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574253.431377:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574253.431390:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574253/real 1788574253] req@ffff9ada604e3480 x1875449616413312/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574278 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574253.431410:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3480 x1875449616413312/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574278 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574253.431418:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574253.431421:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574253.431425:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616413312:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574253.622957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574253.942988:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574254.646977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574254.967077:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574255.670972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574255.990985:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574256.694949:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574257.014983:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574257.718948:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574258.039054:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574258.550990:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574258.551001:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574258.551005:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574258.551013:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6171 00000100:00080000:2.0:1788574258.551022:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574258.551028:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574258.551061:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616413440/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574258.551083:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574258.551086:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574258.551087:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574258.551090:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6171 00000100:00080000:2.0:1788574258.551093:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574258.551095:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574258.551103:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574258.551110:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616413568/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574258.551117:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574258.551120:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574258.551121:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574258.551124:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6171 00000100:00080000:2.0:1788574258.551126:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574258.551128:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574258.551132:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574258.551138:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616413696/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788574258.551144:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616413440:192.168.202.113@tcp:250:kworker.0 00000800:00000100:1.0:1788574258.551209:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574258.551245:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574258/real 1788574258] req@ffff9ada60625180 x1875449616413440/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574283 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574258.551264:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616413440/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574283 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574258.551275:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574258.551279:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574258.551283:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616413440:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574258.551291:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616413568:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574258.551316:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574258.551330:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574258/real 1788574258] req@ffff9ada60626d80 x1875449616413568/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574283 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574258.551344:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616413568/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574283 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574258.551354:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574258.551357:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574258.551361:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616413568:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574258.551396:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616413696:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574258.551417:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574258.551429:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574258/real 1788574258] req@ffff9ada60627b80 x1875449616413696/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574283 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574258.551442:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616413696/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574283 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574258.551450:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574258.551454:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574258.551457:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616413696:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574258.742941:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574259.062991:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574259.766970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574260.086987:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574260.791007:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574261.110996:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574261.814946:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574262.135243:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574262.838978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574263.159018:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788574263.670998:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574263.671006:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788574263.671010:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574263.671019:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6176 00000100:00080000:1.0:1788574263.671028:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574263.671034:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574263.671067:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ed80 x1875449616413824/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574263.671092:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574263.671096:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788574263.671098:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574263.671102:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6176 00000100:00080000:1.0:1788574263.671106:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574263.671110:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574263.671119:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574263.671132:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566dc00 x1875449616413952/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574263.671138:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616413824:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788574263.671143:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574263.671146:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788574263.671147:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574263.671151:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6176 00000100:00080000:1.0:1788574263.671154:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574263.671158:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574263.671162:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574263.671170:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f800 x1875449616414080/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574263.671196:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574263.671230:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574263/real 1788574263] req@ffff9ada4566ed80 x1875449616413824/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574288 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574263.671250:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ed80 x1875449616413824/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574288 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574263.671261:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574263.671265:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574263.671270:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616413824:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574263.671292:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616413952:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574263.671311:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574263.671320:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574263/real 1788574263] req@ffff9ada4566dc00 x1875449616413952/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574288 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574263.671329:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566dc00 x1875449616413952/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574288 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574263.671335:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574263.671337:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574263.671339:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616413952:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574263.671342:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616414080:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574263.671353:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574263.671363:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574263/real 1788574263] req@ffff9ada4566f800 x1875449616414080/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574288 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574263.671375:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f800 x1875449616414080/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574288 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574263.671384:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574263.671387:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574263.671390:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616414080:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574263.862942:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574264.183041:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574264.886976:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574265.207025:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574265.910924:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574266.231012:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574266.934963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574267.255026:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574267.958964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574268.279019:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788574268.790954:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574268.790962:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788574268.790965:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574268.790980:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6181 00000100:00080000:1.0:1788574268.790989:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574268.790995:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574268.791028:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ea00 x1875449616414208/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574268.791050:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574268.791053:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788574268.791054:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574268.791057:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6181 00000100:00080000:1.0:1788574268.791059:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574268.791062:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574268.791069:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574268.791078:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e300 x1875449616414336/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574268.791087:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574268.791089:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788574268.791091:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574268.791095:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6181 00000100:00100000:3.0:1788574268.791097:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616414208:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788574268.791098:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574268.791103:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574268.791109:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574268.791117:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f480 x1875449616414464/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574268.791156:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574268.791186:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574268/real 1788574268] req@ffff9ada4566ea00 x1875449616414208/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574293 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574268.791206:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ea00 x1875449616414208/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574293 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574268.791217:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574268.791222:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574268.791226:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616414208:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574268.791249:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616414336:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574268.791268:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574268.791287:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574268/real 1788574268] req@ffff9ada4566e300 x1875449616414336/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574293 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574268.791300:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e300 x1875449616414336/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574293 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574268.791306:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574268.791309:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574268.791311:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616414336:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574268.791315:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616414464:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574268.791328:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574268.791334:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574268/real 1788574268] req@ffff9ada4566f480 x1875449616414464/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574293 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574268.791342:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f480 x1875449616414464/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574293 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574268.791352:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574268.791354:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574268.791356:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616414464:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574268.982963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574269.303031:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574270.006957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574270.326979:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574271.030982:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574271.351002:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574272.054964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574272.374978:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574273.078979:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574273.398997:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574273.910988:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574273.910996:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574273.911000:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574273.911008:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6186 00000100:00080000:2.0:1788574273.911017:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574273.911023:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574273.911055:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627800 x1875449616414592/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574273.911092:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574273.911096:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574273.911098:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574273.911102:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6186 00000100:00080000:2.0:1788574273.911106:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574273.911111:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574273.911122:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574273.911136:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626300 x1875449616414720/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574273.911142:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616414592:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788574273.911157:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574273.911160:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574273.911161:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574273.911165:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6186 00000100:00080000:2.0:1788574273.911169:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574273.911171:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574273.911176:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574273.911185:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616414848/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574273.911207:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574273.911238:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574273/real 1788574273] req@ffff9ada60627800 x1875449616414592/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574298 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574273.911253:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627800 x1875449616414592/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574298 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574273.911260:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574273.911263:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574273.911267:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616414592:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574273.911287:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616414720:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574273.911311:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574273.911324:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574273/real 1788574273] req@ffff9ada60626300 x1875449616414720/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574298 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574273.911338:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626300 x1875449616414720/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574298 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574273.911345:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574273.911347:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574273.911350:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616414720:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574273.911354:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616414848:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574273.911369:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574273.911376:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574273/real 1788574273] req@ffff9ada60627b80 x1875449616414848/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574298 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574273.911384:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616414848/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574298 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574273.911389:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574273.911391:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574273.911393:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616414848:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574274.102988:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574274.423039:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574275.126980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574275.447053:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574276.150948:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574276.470985:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574277.174954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574277.494975:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574278.198927:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574278.518967:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574279.030979:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574279.030990:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574279.030993:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574279.031003:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6191 00000100:00080000:2.0:1788574279.031014:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574279.031021:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574279.031056:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616414976/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574279.031085:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574279.031089:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574279.031091:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574279.031094:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6191 00000100:00080000:2.0:1788574279.031099:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574279.031104:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574279.031114:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574279.031131:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616415104/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574279.031143:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1788574279.031149:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616414976:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788574279.031150:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574279.031156:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574279.031168:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6191 00000100:00080000:2.0:1788574279.031171:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574279.031176:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574279.031183:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574279.031197:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616415232/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574279.031235:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574279.031283:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574279/real 1788574279] req@ffff9ada60626d80 x1875449616414976/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574304 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574279.031309:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616414976/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574304 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574279.031319:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574279.031323:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574279.031327:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616414976:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574279.031354:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616415104:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574279.031374:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574279.031383:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574279/real 1788574279] req@ffff9ada60625180 x1875449616415104/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574304 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574279.031392:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616415104/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574304 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574279.031399:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574279.031401:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574279.031403:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616415104:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574279.031407:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616415232:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574279.031420:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574279.031431:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574279/real 1788574279] req@ffff9ada60627480 x1875449616415232/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574304 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574279.031444:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616415232/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574304 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574279.031461:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574279.031465:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574279.031468:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616415232:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574279.222969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574279.543040:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574280.246994:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574280.567035:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574281.270937:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574281.590953:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574282.294971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574282.615004:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574283.318927:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574283.639034:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788574284.150960:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574284.150969:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788574284.150973:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574284.150981:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6196 00000100:00080000:1.0:1788574284.150991:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574284.150997:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574284.151038:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d500 x1875449616415360/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574284.151060:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574284.151062:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788574284.151063:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574284.151066:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6196 00000100:00080000:1.0:1788574284.151068:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574284.151071:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574284.151079:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574284.151087:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ca80 x1875449616415488/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574284.151094:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574284.151097:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788574284.151098:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574284.151101:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6196 00000100:00080000:1.0:1788574284.151103:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574284.151106:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574284.151109:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574284.151112:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616415360:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574284.151118:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d180 x1875449616415616/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574284.151169:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574284.151200:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574284/real 1788574284] req@ffff9ada4566d500 x1875449616415360/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574309 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574284.151214:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d500 x1875449616415360/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574309 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574284.151222:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574284.151225:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574284.151228:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616415360:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574284.151233:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616415488:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574284.151254:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574284.151262:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574284/real 1788574284] req@ffff9ada4566ca80 x1875449616415488/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574309 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574284.151270:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ca80 x1875449616415488/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574309 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574284.151276:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574284.151278:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574284.151280:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616415488:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574284.151299:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616415616:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574284.151310:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574284.151315:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574284/real 1788574284] req@ffff9ada4566d180 x1875449616415616/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574309 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574284.151323:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d180 x1875449616415616/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574309 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574284.151328:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574284.151329:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574284.151331:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616415616:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574284.342967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574284.662983:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574285.366922:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574285.687039:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574286.390922:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574286.711046:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574287.414947:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574287.734992:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574288.438980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574288.758995:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788574289.270966:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574289.270975:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788574289.270980:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574289.270991:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6201 00000100:00080000:1.0:1788574289.271002:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574289.271009:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574289.271045:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f100 x1875449616415744/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574289.271070:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574289.271074:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788574289.271080:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574289.271084:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6201 00000100:00080000:1.0:1788574289.271087:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574289.271090:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574289.271099:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574289.271114:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c380 x1875449616415872/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574289.271117:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616415744:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788574289.271133:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574289.271137:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788574289.271139:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574289.271144:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6201 00000100:00080000:1.0:1788574289.271147:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574289.271152:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574289.271157:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574289.271168:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c700 x1875449616416000/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574289.271177:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574289.271209:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574289/real 1788574289] req@ffff9ada4566f100 x1875449616415744/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574314 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574289.271223:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f100 x1875449616415744/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574314 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574289.271232:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574289.271235:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574289.271237:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616415744:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574289.271255:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616415872:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574289.271269:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574289.271286:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574289/real 1788574289] req@ffff9ada4566c380 x1875449616415872/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574314 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574289.271295:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c380 x1875449616415872/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574314 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574289.271300:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574289.271303:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574289.271305:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616415872:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574289.271309:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616416000:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574289.271320:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574289.271326:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574289/real 1788574289] req@ffff9ada4566c700 x1875449616416000/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574314 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574289.271365:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c700 x1875449616416000/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574314 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574289.271371:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574289.271373:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574289.271375:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616416000:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574289.462967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574289.783030:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574290.486952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574290.806972:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574291.510949:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574291.831051:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574292.534964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574292.855003:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574293.558957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574293.879047:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574294.390958:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574294.390969:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574294.390973:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574294.390990:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6206 00000100:00080000:2.0:1788574294.391003:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574294.391011:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574294.391058:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616416128/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574294.391081:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574294.391084:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574294.391086:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574294.391089:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6206 00000100:00080000:2.0:1788574294.391092:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574294.391094:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574294.391105:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574294.391115:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616416256/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574294.391124:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1788574294.391126:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616416128:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788574294.391128:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574294.391132:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574294.391137:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6206 00000100:00080000:2.0:1788574294.391140:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574294.391142:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574294.391145:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574294.391155:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616416384/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574294.391203:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574294.391240:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574294/real 1788574294] req@ffff9ada60624e00 x1875449616416128/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574319 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574294.391274:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616416128/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574319 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574294.391283:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574294.391287:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574294.391290:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616416128:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574294.391316:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616416256:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574294.391335:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574294.391344:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574294/real 1788574294] req@ffff9ada60625f80 x1875449616416256/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574319 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574294.391353:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616416256/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574319 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574294.391359:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574294.391361:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574294.391363:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616416256:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574294.391367:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616416384:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574294.391377:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574294.391383:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574294/real 1788574294] req@ffff9ada60625c00 x1875449616416384/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574319 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574294.391390:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616416384/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574319 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574294.391395:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574294.391397:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574294.391399:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616416384:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574294.582974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574294.903006:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574295.606953:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574295.927008:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574296.630971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574296.950991:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574297.654950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574297.975019:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574298.678977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574298.999098:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574299.510954:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574299.510965:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574299.510970:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574299.510980:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6212 00000100:00080000:0.0:1788574299.510992:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574299.511000:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574299.511045:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616416512/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574299.511069:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574299.511072:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574299.511073:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574299.511076:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6212 00000100:00080000:0.0:1788574299.511079:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574299.511081:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574299.511091:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574299.511100:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616416640/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574299.511107:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574299.511110:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574299.511112:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574299.511114:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6212 00000100:00080000:0.0:1788574299.511116:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574299.511119:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574299.511124:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574299.511133:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d500 x1875449616416768/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574299.511155:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616416512:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788574299.511221:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574299.511270:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574299/real 1788574299] req@ffff9ada4557ea00 x1875449616416512/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574324 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574299.511286:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616416512/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574324 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574299.511295:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574299.511298:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574299.511301:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616416512:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574299.511307:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616416640:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574299.511324:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574299.511332:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574299/real 1788574299] req@ffff9ada4557e680 x1875449616416640/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574324 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574299.511340:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616416640/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574324 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574299.511345:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574299.511347:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574299.511349:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616416640:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574299.511369:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616416768:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574299.511383:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574299.511389:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574299/real 1788574299] req@ffff9ada4557d500 x1875449616416768/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574324 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574299.511404:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d500 x1875449616416768/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574324 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574299.511410:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574299.511412:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574299.511414:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616416768:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574299.702969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574300.022988:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574300.726955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574301.047142:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574301.751020:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574302.071046:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574302.774977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574303.095046:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574303.798977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574304.119064:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574304.630946:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574304.630956:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574304.630961:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574304.630972:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6217 00000100:00080000:0.0:1788574304.630984:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574304.630993:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574304.631046:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d180 x1875449616416896/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574304.631076:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574304.631079:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574304.631080:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574304.631084:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6217 00000100:00080000:0.0:1788574304.631086:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574304.631089:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574304.631098:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574304.631106:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557df80 x1875449616417024/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574304.631114:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574304.631116:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574304.631117:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574304.631119:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6217 00000100:00080000:0.0:1788574304.631122:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574304.631125:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574304.631128:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574304.631136:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d500 x1875449616417152/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574304.631197:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616416896:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788574304.631264:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574304.631321:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574304/real 1788574304] req@ffff9ada4557d180 x1875449616416896/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574329 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574304.631337:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d180 x1875449616416896/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574329 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574304.631346:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574304.631349:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574304.631354:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616416896:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574304.631359:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616417024:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574304.631376:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574304.631384:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574304/real 1788574304] req@ffff9ada4557df80 x1875449616417024/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574329 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574304.631391:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557df80 x1875449616417024/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574329 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574304.631396:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574304.631398:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574304.631400:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616417024:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574304.631404:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616417152:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574304.631414:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574304.631420:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574304/real 1788574304] req@ffff9ada4557d500 x1875449616417152/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574329 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574304.631427:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d500 x1875449616417152/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574329 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574304.631432:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574304.631434:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574304.631436:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616417152:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574304.823093:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574305.143099:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574305.846981:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574306.166997:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574306.871000:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574307.191094:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574307.894958:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574308.215021:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574308.918983:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574309.239077:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574309.751008:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574309.751018:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574309.751024:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574309.751035:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6222 00000100:00080000:2.0:1788574309.751047:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574309.751054:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574309.751099:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616417280/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574309.751133:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574309.751136:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574309.751138:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574309.751141:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6222 00000100:00080000:2.0:1788574309.751143:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574309.751146:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574309.751155:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574309.751168:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616417408/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574309.751176:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574309.751179:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574309.751181:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574309.751184:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6222 00000100:00080000:2.0:1788574309.751186:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574309.751188:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574309.751193:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574309.751203:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616417280:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788574309.751204:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616417536/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574309.751294:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574309.751343:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574309/real 1788574309] req@ffff9ada60625500 x1875449616417280/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574334 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574309.751363:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616417280/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574334 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574309.751372:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574309.751376:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574309.751379:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616417280:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574309.751384:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616417408:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574309.751404:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574309.751413:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574309/real 1788574309] req@ffff9ada60625880 x1875449616417408/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574334 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574309.751422:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616417408/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574334 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574309.751428:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574309.751430:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574309.751432:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616417408:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574309.751457:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616417536:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574309.751470:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574309.751476:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574309/real 1788574309] req@ffff9ada60624a80 x1875449616417536/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574334 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574309.751484:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616417536/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574334 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574309.751496:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574309.751498:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574309.751500:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616417536:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574309.942967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574310.263038:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574310.967004:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574311.287093:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574311.990953:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574312.310995:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574313.014977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574313.335067:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574314.038999:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574314.359049:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788574314.871006:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574314.871017:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788574314.871021:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574314.871032:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6227 00000100:00080000:3.0:1788574314.871043:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574314.871052:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574314.871095:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3480 x1875449616417664/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574314.871158:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574314.871163:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788574314.871165:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574314.871169:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6227 00000100:00080000:3.0:1788574314.871173:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574314.871178:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574314.871188:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574314.871203:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0700 x1875449616417792/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574314.871216:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1788574314.871218:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616417664:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788574314.871222:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788574314.871227:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574314.871232:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6227 00000100:00080000:3.0:1788574314.871235:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574314.871240:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574314.871245:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574314.871259:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0380 x1875449616417920/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788574314.871312:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616417792:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574314.871337:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616417920:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574315.062976:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574315.383013:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574316.086946:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574316.407099:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574317.110962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574317.431047:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1788574317.943201:0:2373:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:2.0:1788574317.943253:0:2373:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1788574317.943270:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788574317.943329:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788574317.943341:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:1.0:1788574317.943400:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574314/real 1788574317] req@ffff9ada604e0700 x1875449616417792/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574339 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574317.943465:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0700 x1875449616417792/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574339 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574317.943481:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574317.943487:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574317.943492:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616417792:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574317.943502:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574314/real 1788574317] req@ffff9ada604e0380 x1875449616417920/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574339 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574317.943514:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0380 x1875449616417920/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574339 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574317.943522:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574317.943528:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574317.943531:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616417920:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574317.943539:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574314/real 1788574317] req@ffff9ada604e3480 x1875449616417664/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574339 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574317.943552:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3480 x1875449616417664/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574339 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574317.943561:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574317.943566:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574317.943569:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616417664:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574318.134979:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574318.455061:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574319.158980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574319.479263:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574319.991072:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574319.991084:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574319.991088:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574319.991099:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6232 00000100:00080000:2.0:1788574319.991110:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574319.991118:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574319.991158:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616418048/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574319.991185:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574319.991188:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574319.991190:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574319.991193:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6232 00000100:00080000:2.0:1788574319.991195:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574319.991198:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574319.991210:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574319.991220:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616418176/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574319.991230:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574319.991232:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574319.991234:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574319.991238:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6232 00000100:00080000:2.0:1788574319.991240:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574319.991244:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574319.991251:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574319.991253:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616418048:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788574319.991270:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616418304/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788574319.991337:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574319.991381:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574319/real 1788574319] req@ffff9ada60626a00 x1875449616418048/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574344 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574319.991402:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616418048/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574344 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574319.991413:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574319.991418:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574319.991423:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616418048:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574319.991445:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616418176:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574319.991473:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574319.991487:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574319/real 1788574319] req@ffff9ada60624380 x1875449616418176/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574344 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574319.991500:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616418176/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574344 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574319.991509:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574319.991512:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574319.991515:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616418176:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574319.991545:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616418304:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574319.991568:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574319.991580:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574319/real 1788574319] req@ffff9ada60626680 x1875449616418304/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574344 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574319.991592:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616418304/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574344 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574319.991601:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574319.991605:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574319.991608:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616418304:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574320.182937:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574320.503059:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574321.206989:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574321.526981:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574322.230960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574322.551046:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574323.254967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574323.575005:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574324.279006:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574324.598986:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574325.110958:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574325.110966:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574325.110969:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574325.110978:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6237 00000100:00080000:2.0:1788574325.110986:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574325.110993:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574325.111025:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624700 x1875449616418432/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574325.111048:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574325.111051:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574325.111053:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574325.111056:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6237 00000100:00080000:2.0:1788574325.111058:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574325.111060:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574325.111068:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574325.111076:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627100 x1875449616418560/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574325.111083:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574325.111086:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574325.111087:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574325.111089:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6237 00000100:00080000:2.0:1788574325.111091:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574325.111094:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574325.111098:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574325.111110:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616418688/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788574325.111132:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616418432:192.168.202.113@tcp:250:kworker.0 00000800:00000100:1.0:1788574325.111186:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574325.111215:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574325/real 1788574325] req@ffff9ada60624700 x1875449616418432/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574350 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574325.111229:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624700 x1875449616418432/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574350 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574325.111237:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574325.111240:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574325.111243:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616418432:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574325.111248:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616418560:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574325.111261:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574325.111267:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574325/real 1788574325] req@ffff9ada60627100 x1875449616418560/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574350 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574325.111275:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627100 x1875449616418560/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574350 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574325.111280:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574325.111282:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574325.111284:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616418560:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574325.111287:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616418688:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574325.111297:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574325.111302:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574325/real 1788574325] req@ffff9ada60626680 x1875449616418688/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574350 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574325.111309:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616418688/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574350 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574325.111314:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574325.111316:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574325.111318:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616418688:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574325.302972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574325.623028:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574326.326941:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574326.647113:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574327.350920:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574327.670964:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574328.374951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574328.695048:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574329.398970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574329.718981:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574330.230964:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574330.230972:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574330.230975:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574330.230984:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6242 00000100:00080000:2.0:1788574330.230993:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574330.230999:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574330.231032:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616418816/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574330.231057:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574330.231061:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574330.231063:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574330.231067:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6242 00000100:00080000:2.0:1788574330.231070:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574330.231074:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574330.231083:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574330.231098:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616418944/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788574330.231110:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616418816:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788574330.231121:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574330.231125:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574330.231127:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574330.231130:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6242 00000100:00080000:2.0:1788574330.231134:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574330.231137:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574330.231141:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574330.231152:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616419072/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788574330.231184:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574330.231221:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574330/real 1788574330] req@ffff9ada60624380 x1875449616418816/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574355 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574330.231252:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616418816/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574355 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574330.231264:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574330.231268:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574330.231272:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616418816:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574330.231297:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616418944:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574330.231321:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574330.231336:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574330/real 1788574330] req@ffff9ada60626a00 x1875449616418944/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574355 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574330.231349:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616418944/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574355 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574330.231358:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574330.231362:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574330.231366:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616418944:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574330.231376:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616419072:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574330.231397:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574330.231409:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574330/real 1788574330] req@ffff9ada60624a80 x1875449616419072/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574355 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574330.231423:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616419072/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574355 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574330.231431:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574330.231434:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574330.231438:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616419072:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574330.422952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574330.742985:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574331.446992:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574331.767084:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574332.470985:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574332.791088:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574333.494934:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574333.815184:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574334.518933:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574334.838989:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788574335.350965:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574335.350973:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788574335.350977:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574335.350985:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6247 00000100:00080000:1.0:1788574335.350993:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574335.350999:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574335.351029:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ce00 x1875449616419200/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574335.351051:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574335.351054:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788574335.351055:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574335.351058:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6247 00000100:00080000:1.0:1788574335.351061:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574335.351063:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574335.351070:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574335.351079:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d880 x1875449616419328/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574335.351086:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574335.351089:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788574335.351090:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574335.351093:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6247 00000100:00080000:1.0:1788574335.351095:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574335.351097:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574335.351102:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574335.351103:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616419200:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574335.351116:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e680 x1875449616419456/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574335.351174:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574335.351208:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574335/real 1788574335] req@ffff9ada4566ce00 x1875449616419200/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574360 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574335.351229:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ce00 x1875449616419200/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574360 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574335.351241:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574335.351245:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574335.351249:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616419200:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574335.351257:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616419328:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574335.351281:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574335.351294:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574335/real 1788574335] req@ffff9ada4566d880 x1875449616419328/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574360 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574335.351307:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d880 x1875449616419328/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574360 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574335.351317:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574335.351320:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574335.351324:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616419328:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574335.351347:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616419456:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574335.351367:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574335.351378:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574335/real 1788574335] req@ffff9ada4566e680 x1875449616419456/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574360 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574335.351390:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e680 x1875449616419456/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574360 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574335.351400:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574335.351403:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574335.351407:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616419456:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574335.542967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574335.863052:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574336.566989:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574336.887010:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574337.590999:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574337.911007:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574338.614962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574338.934994:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574339.638916:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574339.958996:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574340.470949:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574340.470961:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574340.470966:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574340.470978:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6253 00000100:00080000:2.0:1788574340.470988:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574340.470996:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574340.471034:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616419584/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574340.471063:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574340.471067:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574340.471069:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574340.471073:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6253 00000100:00080000:2.0:1788574340.471078:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574340.471083:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574340.471094:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574340.471103:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616419584:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788574340.471110:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616419712/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574340.471123:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574340.471127:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574340.471130:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574340.471134:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6253 00000100:00080000:2.0:1788574340.471150:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574340.471155:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000800:00000100:3.0:1788574340.471158:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000008:00100000:2.0:1788574340.471163:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574340.471179:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616419840/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574340.471192:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574340/real 1788574340] req@ffff9ada60625880 x1875449616419584/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574365 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574340.471206:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616419584/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574365 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574340.471214:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574340.471217:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574340.471220:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616419584:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574340.471238:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616419712:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574340.471252:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574340.471260:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574340/real 1788574340] req@ffff9ada60625500 x1875449616419712/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574365 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574340.471268:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616419712/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574365 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574340.471273:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574340.471275:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574340.471277:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616419712:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574340.471290:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616419840:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574340.471302:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574340.471309:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574340/real 1788574340] req@ffff9ada60625c00 x1875449616419840/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574365 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574340.471321:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616419840/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574365 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574340.471332:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574340.471336:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574340.471340:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616419840:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574340.662991:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574340.983238:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574341.686933:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574342.007060:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788574342.711024:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574343.030948:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788574343.734942:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574344.055041:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788574344.758970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574345.079001:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574345.590936:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574345.590944:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574345.590947:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574345.590955:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6258 00000100:00080000:2.0:1788574345.590965:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574345.590971:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574345.591002:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616419968/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574345.591026:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574345.591028:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574345.591030:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574345.591033:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6258 00000100:00080000:2.0:1788574345.591035:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574345.591037:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574345.591044:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574345.591051:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616420096/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574345.591058:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574345.591060:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574345.591062:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574345.591065:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6258 00000100:00080000:2.0:1788574345.591067:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574345.591069:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574345.591071:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574345.591077:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616420224/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574345.591103:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616419968:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788574345.591158:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574345.591186:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574345/real 1788574345] req@ffff9ada60625f80 x1875449616419968/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574370 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574345.591200:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616419968/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574370 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574345.591208:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574345.591211:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574345.591214:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616419968:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574345.591218:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616420096:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574345.591231:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574345.591238:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574345/real 1788574345] req@ffff9ada60624e00 x1875449616420096/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574370 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574345.591246:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616420096/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574370 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574345.591252:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574345.591254:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574345.591256:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616420096:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574345.591259:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616420224:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574345.591269:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574345.591274:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574345/real 1788574345] req@ffff9ada60627480 x1875449616420224/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574370 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574345.591281:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616420224/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574370 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574345.591286:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574345.591288:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574345.591290:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616420224:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788574345.782967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574346.102952:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788574346.806960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574347.126970:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788574347.830960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574348.151067:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788574348.854962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574349.174966:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788574349.878945:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574350.199047:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788574350.710999:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574350.711009:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788574350.711013:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574350.711022:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6263 00000100:00080000:1.0:1788574350.711032:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574350.711040:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574350.711083:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e680 x1875449616420352/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574350.711113:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574350.711116:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788574350.711118:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574350.711121:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6263 00000100:00080000:1.0:1788574350.711124:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574350.711127:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574350.711134:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574350.711145:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d880 x1875449616420480/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574350.711154:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574350.711158:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1788574350.711158:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616420352:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788574350.711162:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574350.711169:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6263 00000100:00080000:1.0:1788574350.711172:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574350.711174:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574350.711177:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574350.711185:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ce00 x1875449616420608/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574350.711223:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574350.711259:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574350/real 1788574350] req@ffff9ada4566e680 x1875449616420352/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574375 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574350.711273:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e680 x1875449616420352/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574375 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574350.711281:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574350.711284:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574350.711287:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616420352:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574350.711306:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616420480:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574350.711323:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574350.711331:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574350/real 1788574350] req@ffff9ada4566d880 x1875449616420480/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574375 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574350.711349:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d880 x1875449616420480/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574375 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574350.711355:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574350.711357:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574350.711359:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616420480:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574350.711363:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616420608:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574350.711375:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574350.711381:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574350/real 1788574350] req@ffff9ada4566ce00 x1875449616420608/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574375 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574350.711388:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ce00 x1875449616420608/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574375 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574350.711393:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574350.711396:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574350.711398:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616420608:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788574350.902980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574351.223005:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788574351.926951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574352.246984:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788574352.950978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574353.271060:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788574353.974930:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574354.295047:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788574354.998969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574355.319100:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788574355.830977:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574355.830986:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788574355.830990:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574355.830999:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6268 00000100:00080000:1.0:1788574355.831008:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574355.831013:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574355.831045:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c700 x1875449616420736/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574355.831071:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574355.831074:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788574355.831076:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574355.831078:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6268 00000100:00080000:1.0:1788574355.831080:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574355.831083:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574355.831090:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574355.831098:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c380 x1875449616420864/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574355.831106:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574355.831108:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788574355.831109:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574355.831112:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6268 00000100:00080000:1.0:1788574355.831115:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574355.831118:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574355.831122:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574355.831124:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616420736:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574355.831131:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f100 x1875449616420992/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574355.831181:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574355.831208:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574355/real 1788574355] req@ffff9ada4566c700 x1875449616420736/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574380 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574355.831222:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c700 x1875449616420736/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574380 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574355.831231:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574355.831234:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574355.831237:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616420736:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574355.831242:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616420864:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574355.831256:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574355.831272:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574355/real 1788574355] req@ffff9ada4566c380 x1875449616420864/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574380 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574355.831281:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c380 x1875449616420864/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574380 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574355.831287:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574355.831289:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574355.831291:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616420864:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574355.831309:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616420992:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574355.831321:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574355.831335:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574355/real 1788574355] req@ffff9ada4566f100 x1875449616420992/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574380 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574355.831343:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f100 x1875449616420992/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574380 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574355.831353:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574355.831355:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574355.831358:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616420992:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788574356.022951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574356.343072:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574357.047020:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574357.367055:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574358.070969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574358.390975:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574359.094966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574359.415064:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574360.118961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574360.438979:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788574360.950986:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574360.950997:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788574360.951002:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574360.951012:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6273 00000100:00080000:1.0:1788574360.951024:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574360.951032:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574360.951069:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d180 x1875449616421120/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574360.951095:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574360.951098:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788574360.951099:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574360.951102:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6273 00000100:00080000:1.0:1788574360.951105:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574360.951108:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574360.951115:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574360.951123:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ca80 x1875449616421248/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574360.951136:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574360.951139:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788574360.951141:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574360.951146:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6273 00000100:00100000:3.0:1788574360.951147:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616421120:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788574360.951150:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574360.951155:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574360.951158:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574360.951167:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d500 x1875449616421376/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574360.951206:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574360.951235:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574360/real 1788574360] req@ffff9ada4566d180 x1875449616421120/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574385 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574360.951249:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d180 x1875449616421120/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574385 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574360.951258:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574360.951260:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574360.951263:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616421120:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574360.951281:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616421248:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574360.951295:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574360.951303:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574360/real 1788574360] req@ffff9ada4566ca80 x1875449616421248/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574385 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574360.951311:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ca80 x1875449616421248/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574385 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574360.951320:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574360.951323:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574360.951325:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616421248:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574360.951328:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616421376:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574360.951338:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574360.951343:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574360/real 1788574360] req@ffff9ada4566d500 x1875449616421376/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574385 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574360.951351:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d500 x1875449616421376/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574385 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574360.951356:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574360.951358:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574360.951360:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616421376:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574361.142929:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574361.463022:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574362.166999:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574362.487057:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574363.190985:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574363.511062:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574364.214965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574364.534974:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574365.238963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574365.559047:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574366.071033:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574366.071043:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574366.071048:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574366.071060:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6278 00000100:00080000:2.0:1788574366.071072:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574366.071080:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574366.071121:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616421504/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574366.071154:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574366.071158:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574366.071161:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574366.071165:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6278 00000100:00080000:2.0:1788574366.071168:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574366.071173:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574366.071182:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574366.071198:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616421632/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574366.071201:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616421504:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788574366.071218:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574366.071222:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574366.071225:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574366.071229:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6278 00000100:00080000:2.0:1788574366.071234:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574366.071239:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574366.071245:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574366.071258:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616421760/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574366.071269:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574366.071307:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574366/real 1788574366] req@ffff9ada60625180 x1875449616421504/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574391 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574366.071329:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616421504/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574391 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574366.071340:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574366.071345:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574366.071350:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616421504:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574366.071373:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616421632:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574366.071405:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574366.071419:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574366/real 1788574366] req@ffff9ada60626d80 x1875449616421632/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574391 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574366.071433:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616421632/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574391 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574366.071449:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574366.071454:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574366.071457:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616421632:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574366.071464:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616421760:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574366.071486:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574366.071495:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574366/real 1788574366] req@ffff9ada60627b80 x1875449616421760/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574391 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574366.071504:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616421760/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574391 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574366.071509:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574366.071511:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574366.071514:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616421760:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574366.262984:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574366.583059:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574367.286969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574367.607003:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574368.310993:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574368.631006:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574369.334944:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574369.655130:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574370.358989:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574370.679024:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788574371.191032:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574371.191043:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788574371.191047:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574371.191058:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6283 00000100:00080000:1.0:1788574371.191075:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574371.191084:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574371.191128:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f480 x1875449616421888/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574371.191155:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574371.191159:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788574371.191161:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574371.191165:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6283 00000100:00080000:1.0:1788574371.191170:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574371.191174:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574371.191187:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574371.191201:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e300 x1875449616422016/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574371.191211:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574371.191214:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788574371.191216:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1788574371.191216:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616421888:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788574371.191220:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6283 00000100:00080000:1.0:1788574371.191223:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574371.191227:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574371.191231:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574371.191241:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ea00 x1875449616422144/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574371.191298:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574371.191340:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574371/real 1788574371] req@ffff9ada4566f480 x1875449616421888/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574396 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574371.191357:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f480 x1875449616421888/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574396 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574371.191366:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574371.191369:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574371.191373:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616421888:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574371.191395:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616422016:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574371.191412:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574371.191420:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574371/real 1788574371] req@ffff9ada4566e300 x1875449616422016/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574396 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574371.191429:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e300 x1875449616422016/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574396 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574371.191434:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574371.191436:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574371.191440:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616422016:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574371.191444:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616422144:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574371.191463:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574371.191471:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574371/real 1788574371] req@ffff9ada4566ea00 x1875449616422144/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574396 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574371.191478:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ea00 x1875449616422144/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574396 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574371.191483:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574371.191486:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574371.191488:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616422144:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574371.382976:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574371.703086:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574372.406997:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574372.727112:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574373.430963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574373.751041:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574374.454992:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574374.775153:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574375.479001:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574375.799071:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788574376.311031:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574376.311045:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788574376.311051:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574376.311064:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6288 00000100:00080000:1.0:1788574376.311078:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574376.311086:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574376.311130:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f800 x1875449616422272/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574376.311163:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574376.311166:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788574376.311168:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574376.311171:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6288 00000100:00080000:1.0:1788574376.311173:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574376.311176:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574376.311191:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574376.311202:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566dc00 x1875449616422400/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574376.311210:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574376.311213:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788574376.311215:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574376.311218:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6288 00000100:00080000:1.0:1788574376.311221:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574376.311223:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574376.311226:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574376.311234:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ed80 x1875449616422528/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574376.311253:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616422272:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788574376.311329:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574376.311369:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574376/real 1788574376] req@ffff9ada4566f800 x1875449616422272/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574401 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574376.311391:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f800 x1875449616422272/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574401 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574376.311404:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574376.311408:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574376.311414:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616422272:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574376.311422:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616422400:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574376.311444:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574376.311457:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574376/real 1788574376] req@ffff9ada4566dc00 x1875449616422400/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574401 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574376.311472:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566dc00 x1875449616422400/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574401 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574376.311481:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574376.311485:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574376.311489:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616422400:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574376.311514:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616422528:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574376.311534:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574376.311545:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574376/real 1788574376] req@ffff9ada4566ed80 x1875449616422528/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574401 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574376.311558:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ed80 x1875449616422528/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574401 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574376.311567:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574376.311571:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574376.311575:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616422528:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574376.502963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574376.823142:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574377.526962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574377.847050:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574378.550940:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574378.871112:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574379.574997:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574379.895098:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574380.598962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574380.919057:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574381.431054:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574381.431065:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574381.431070:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574381.431083:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6293 00000100:00080000:2.0:1788574381.431098:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574381.431107:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574381.431156:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626300 x1875449616422656/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574381.431181:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574381.431184:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574381.431185:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574381.431188:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6293 00000100:00080000:2.0:1788574381.431190:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574381.431194:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574381.431203:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574381.431211:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627800 x1875449616422784/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574381.431218:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574381.431221:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574381.431223:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574381.431225:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6293 00000100:00080000:2.0:1788574381.431228:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574381.431231:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574381.431235:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574381.431243:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616422912/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574381.431285:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616422656:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574381.431382:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616422784:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574381.431403:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616422912:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574381.623014:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574381.943085:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574382.646923:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574382.967002:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574383.671017:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574383.991040:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0:1788574384.503252:0:2371:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:3.0:1788574384.503298:0:2371:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1788574384.503317:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:3.0:1788574384.503365:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:3.0:1788574384.503376:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:1.0:1788574384.503432:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574381/real 1788574384] req@ffff9ada60626300 x1875449616422656/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574406 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574384.503467:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626300 x1875449616422656/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574406 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574384.503480:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574384.503485:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574384.503489:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616422656:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574384.503498:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574381/real 1788574384] req@ffff9ada60627800 x1875449616422784/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574406 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574384.503511:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627800 x1875449616422784/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574406 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574384.503520:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574384.503524:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574384.503527:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616422784:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574384.503534:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574381/real 1788574384] req@ffff9ada60627b80 x1875449616422912/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574406 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574384.503546:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616422912/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574406 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574384.503554:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574384.503558:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574384.503561:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616422912:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574384.694960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574385.015056:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574385.718973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574386.038995:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574386.550948:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574386.550958:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574386.550962:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574386.550972:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6299 00000100:00080000:2.0:1788574386.550982:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574386.550990:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574386.551026:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616423040/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574386.551057:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574386.551062:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574386.551064:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574386.551069:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6299 00000100:00080000:2.0:1788574386.551073:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574386.551077:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574386.551100:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574386.551116:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616423168/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788574386.551127:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616423040:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788574386.551137:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574386.551142:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574386.551144:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574386.551149:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6299 00000100:00080000:2.0:1788574386.551153:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574386.551158:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574386.551163:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574386.551178:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616423296/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788574386.551206:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574386.551241:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574386/real 1788574386] req@ffff9ada60626d80 x1875449616423040/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574411 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574386.551260:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616423040/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574411 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574386.551272:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574386.551276:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574386.551286:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616423040:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574386.551313:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616423168:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574386.551339:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574386.551353:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574386/real 1788574386] req@ffff9ada60625180 x1875449616423168/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574411 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574386.551367:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616423168/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574411 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574386.551377:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574386.551381:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574386.551384:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616423168:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574386.551391:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616423296:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574386.551410:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574386.551422:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574386/real 1788574386] req@ffff9ada60627480 x1875449616423296/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574411 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574386.551440:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616423296/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574411 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574386.551449:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574386.551452:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574386.551456:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616423296:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574386.742972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574387.063016:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574387.766960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574388.087033:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574388.790936:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574389.110981:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574389.814995:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574390.134969:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574390.838954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574391.159184:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574391.671037:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574391.671053:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574391.671059:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574391.671074:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6304 00000100:00080000:2.0:1788574391.671088:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574391.671098:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574391.671188:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616423424/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574391.671226:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574391.671231:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574391.671233:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574391.671238:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6304 00000100:00080000:2.0:1788574391.671243:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574391.671248:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574391.671261:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574391.671280:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616423552/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574391.671296:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574391.671306:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:1.0:1788574391.671308:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616423424:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788574391.671309:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574391.671322:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6304 00000100:00080000:2.0:1788574391.671329:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574391.671335:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574391.671342:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574391.671361:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616423680/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788574391.671386:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574391.671425:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574391/real 1788574391] req@ffff9ada60624e00 x1875449616423424/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574416 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574391.671441:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616423424/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574416 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574391.671450:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574391.671453:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574391.671456:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616423424:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574391.671476:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616423552:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574391.671491:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574391.671499:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574391/real 1788574391] req@ffff9ada60625f80 x1875449616423552/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574416 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574391.671507:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616423552/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574416 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574391.671518:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574391.671520:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574391.671522:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616423552:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574391.671526:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616423680:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574391.671537:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574391.671543:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574391/real 1788574391] req@ffff9ada60625c00 x1875449616423680/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574416 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574391.671550:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616423680/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574416 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574391.671555:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574391.671557:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574391.671559:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616423680:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574391.862964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574392.183063:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574392.886963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574393.207037:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574393.910961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574394.231077:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574394.934958:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574395.255041:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574395.958943:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574396.279027:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788574396.790960:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574396.790971:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788574396.790976:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574396.790987:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6309 00000100:00080000:3.0:1788574396.790999:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574396.791008:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574396.791047:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1180 x1875449616423808/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574396.791077:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574396.791081:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788574396.791084:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574396.791088:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6309 00000100:00080000:3.0:1788574396.791092:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574396.791097:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574396.791116:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574396.791124:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616423808:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574396.791133:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3800 x1875449616423936/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574396.791147:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574396.791151:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788574396.791153:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574396.791157:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6309 00000100:00080000:3.0:1788574396.791163:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574396.791168:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574396.791174:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000800:00000100:1.0:1788574396.791180:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574396.791202:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1880 x1875449616424064/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788574396.791239:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574396/real 1788574396] req@ffff9ada604e1180 x1875449616423808/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574421 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574396.791254:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1180 x1875449616423808/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574421 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574396.791262:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574396.791265:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574396.791268:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616423808:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574396.791286:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616423936:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574396.791302:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574396.791310:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574396/real 1788574396] req@ffff9ada604e3800 x1875449616423936/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574421 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574396.791318:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3800 x1875449616423936/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574421 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574396.791323:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574396.791325:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574396.791327:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616423936:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574396.791331:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616424064:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574396.791341:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574396.791347:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574396/real 1788574396] req@ffff9ada604e1880 x1875449616424064/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574421 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574396.791359:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1880 x1875449616424064/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574421 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574396.791367:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574396.791370:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574396.791374:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616424064:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574396.982966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574397.303030:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574398.006974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574398.327109:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574399.031015:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574399.351002:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574400.054951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574400.375075:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574401.078989:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574401.399012:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574401.910999:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574401.911008:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574401.911012:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574401.911020:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6314 00000100:00080000:2.0:1788574401.911029:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574401.911035:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574401.911069:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616424192/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574401.911097:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574401.911099:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574401.911101:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574401.911103:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6314 00000100:00080000:2.0:1788574401.911105:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574401.911108:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574401.911115:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574401.911125:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616424320/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574401.911136:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1788574401.911140:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616424192:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788574401.911142:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574401.911147:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574401.911154:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6314 00000100:00080000:2.0:1788574401.911158:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574401.911162:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574401.911168:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574401.911184:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616424448/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788574401.911200:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574401.911231:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574401/real 1788574401] req@ffff9ada60625500 x1875449616424192/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574426 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574401.911271:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616424192/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574426 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574401.911291:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574401.911295:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574401.911299:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616424192:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574401.911330:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616424320:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574401.911351:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574401.911362:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574401/real 1788574401] req@ffff9ada60625880 x1875449616424320/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574426 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574401.911372:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616424320/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574426 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574401.911381:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574401.911384:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574401.911398:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616424320:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574401.911405:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616424448:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574401.911427:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574401.911436:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574401/real 1788574401] req@ffff9ada60624a80 x1875449616424448/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574426 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574401.911445:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616424448/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574426 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574401.911455:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574401.911457:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574401.911459:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616424448:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574402.102982:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574402.423036:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574403.126973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574403.447023:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574404.150987:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574404.471023:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574405.174979:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574405.495031:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574406.199012:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574406.519107:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788574407.030986:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574407.030998:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788574407.031003:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574407.031014:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6319 00000100:00080000:1.0:1788574407.031026:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574407.031041:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574407.031093:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c000 x1875449616424576/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574407.031125:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574407.031130:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788574407.031132:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574407.031136:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6319 00000100:00080000:1.0:1788574407.031141:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574407.031146:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574407.031159:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574407.031175:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566df80 x1875449616424704/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574407.031185:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616424576:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788574407.031187:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574407.031195:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788574407.031197:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574407.031201:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6319 00000100:00080000:1.0:1788574407.031204:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574407.031207:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574407.031212:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574407.031227:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566fb80 x1875449616424832/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574407.031276:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574407.031347:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574407/real 1788574407] req@ffff9ada4566c000 x1875449616424576/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574432 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574407.031371:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c000 x1875449616424576/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574432 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574407.031384:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574407.031388:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574407.031391:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616424576:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574407.031417:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616424704:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574407.031445:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574407.031454:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574407/real 1788574407] req@ffff9ada4566df80 x1875449616424704/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574432 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574407.031463:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566df80 x1875449616424704/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574432 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574407.031470:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574407.031473:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574407.031475:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616424704:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574407.031479:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616424832:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574407.031490:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574407.031496:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574407/real 1788574407] req@ffff9ada4566fb80 x1875449616424832/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574432 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574407.031504:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566fb80 x1875449616424832/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574432 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574407.031509:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574407.031511:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574407.031512:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616424832:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574407.222990:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574407.542981:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574408.246945:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574408.567047:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574409.270969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574409.591109:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574410.294972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574410.615022:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574411.318963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574411.639051:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788574412.150959:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574412.150970:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788574412.150975:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574412.150987:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6324 00000100:00080000:1.0:1788574412.151000:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574412.151010:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574412.151065:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566fb80 x1875449616424960/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574412.151097:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574412.151100:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788574412.151102:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574412.151106:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6324 00000100:00080000:1.0:1788574412.151109:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574412.151111:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574412.151121:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574412.151132:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566df80 x1875449616425088/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574412.151146:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574412.151151:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1788574412.151152:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616424960:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788574412.151153:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574412.151162:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6324 00000100:00080000:1.0:1788574412.151165:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574412.151167:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574412.151172:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574412.151193:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c000 x1875449616425216/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574412.151230:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574412.151267:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574412/real 1788574412] req@ffff9ada4566fb80 x1875449616424960/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574437 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574412.151294:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566fb80 x1875449616424960/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574437 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574412.151304:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574412.151307:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574412.151310:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616424960:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574412.151333:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616425088:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574412.151367:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574412.151378:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574412/real 1788574412] req@ffff9ada4566df80 x1875449616425088/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574437 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574412.151390:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566df80 x1875449616425088/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574437 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574412.151399:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574412.151403:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574412.151406:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616425088:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574412.151413:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616425216:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574412.151435:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574412.151447:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574412/real 1788574412] req@ffff9ada4566c000 x1875449616425216/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574437 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574412.151456:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c000 x1875449616425216/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574437 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574412.151462:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574412.151464:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574412.151466:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616425216:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574412.342951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574412.663059:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574413.366970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574413.687024:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574414.390980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574414.711010:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574415.414960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574415.735078:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574416.438999:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574416.758953:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574417.270973:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574417.270989:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574417.270995:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574417.271009:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6329 00000100:00080000:0.0:1788574417.271024:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574417.271033:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574417.271090:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616425344/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574417.271122:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574417.271126:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574417.271127:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574417.271131:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6329 00000100:00080000:0.0:1788574417.271134:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574417.271138:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574417.271148:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574417.271160:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616425472/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574417.271169:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574417.271173:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574417.271176:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1788574417.271176:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616425344:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788574417.271185:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6329 00000100:00080000:0.0:1788574417.271188:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574417.271191:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574417.271194:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574417.271204:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616425600/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574417.271254:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574417.271294:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574417/real 1788574417] req@ffff9ada4557e680 x1875449616425344/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574442 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574417.271312:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616425344/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574442 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574417.271323:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574417.271328:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574417.271332:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616425344:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574417.271363:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616425472:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574417.271386:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574417.271422:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574417/real 1788574417] req@ffff9ada4557ea00 x1875449616425472/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574442 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574417.271432:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616425472/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574442 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574417.271439:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574417.271441:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574417.271443:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616425472:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574417.271449:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616425600:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574417.271465:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574417.271472:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574417/real 1788574417] req@ffff9ada4557ed80 x1875449616425600/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574442 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574417.271481:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616425600/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574442 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574417.271486:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574417.271488:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574417.271491:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616425600:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574417.462948:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574417.783058:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574418.486980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574418.807086:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574419.510943:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574419.831156:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574420.534966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574420.854978:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574421.558915:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574421.879060:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574422.390944:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574422.390955:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574422.390962:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574422.390976:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6334 00000100:00080000:2.0:1788574422.390990:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574422.390999:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574422.391050:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616425728/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574422.391084:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574422.391089:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574422.391091:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574422.391095:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6334 00000100:00080000:2.0:1788574422.391101:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574422.391107:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574422.391120:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574422.391130:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616425728:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788574422.391142:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616425856/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574422.391161:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574422.391165:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574422.391168:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574422.391173:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6334 00000100:00080000:2.0:1788574422.391192:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1788574422.391199:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00080000:2.0:1788574422.391201:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574422.391211:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574422.391232:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616425984/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574422.391240:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574422/real 1788574422] req@ffff9ada60626a00 x1875449616425728/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574447 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574422.391272:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616425728/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574447 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574422.391293:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574422.391298:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574422.391301:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616425728:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574422.391323:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616425856:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574422.391340:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574422.391349:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574422/real 1788574422] req@ffff9ada60624380 x1875449616425856/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574447 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574422.391357:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616425856/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574447 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574422.391364:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574422.391366:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574422.391368:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616425856:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574422.391372:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616425984:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574422.391382:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574422.391388:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574422/real 1788574422] req@ffff9ada60626680 x1875449616425984/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574447 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574422.391396:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616425984/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574447 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574422.391401:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574422.391403:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574422.391405:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616425984:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574422.582970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574422.903016:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574423.606958:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574423.927107:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574424.630953:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574424.951050:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574425.654944:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574425.975005:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574426.678956:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574426.999057:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574427.510976:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574427.510987:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574427.510991:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574427.511003:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6340 00000100:00080000:2.0:1788574427.511016:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574427.511026:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574427.511071:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627100 x1875449616426112/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574427.511100:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574427.511103:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574427.511104:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574427.511108:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6340 00000100:00080000:2.0:1788574427.511111:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574427.511115:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574427.511127:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574427.511141:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624700 x1875449616426240/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574427.511151:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574427.511154:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574427.511157:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574427.511162:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6340 00000100:00080000:2.0:1788574427.511170:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1788574427.511170:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616426112:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788574427.511178:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574427.511193:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574427.511209:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616426368/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574427.511251:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574427.511298:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574427/real 1788574427] req@ffff9ada60627100 x1875449616426112/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574452 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574427.511315:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627100 x1875449616426112/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574452 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574427.511324:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574427.511327:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574427.511330:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616426112:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574427.511351:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616426240:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574427.511367:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574427.511375:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574427/real 1788574427] req@ffff9ada60624700 x1875449616426240/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574452 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574427.511383:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624700 x1875449616426240/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574452 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574427.511389:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574427.511391:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574427.511393:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616426240:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574427.511397:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616426368:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574427.511407:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574427.511413:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574427/real 1788574427] req@ffff9ada60626680 x1875449616426368/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574452 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574427.511421:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616426368/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574452 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574427.511426:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574427.511428:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574427.511430:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616426368:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574427.702939:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574428.023077:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574428.726964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574429.046956:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574429.750911:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574430.071018:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574430.774945:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574431.095041:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574431.798960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574432.118988:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574432.631025:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574432.631038:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574432.631042:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574432.631054:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6345 00000100:00080000:0.0:1788574432.631066:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574432.631074:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574432.631128:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616426496/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574432.631163:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574432.631166:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574432.631168:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574432.631171:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6345 00000100:00080000:0.0:1788574432.631175:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574432.631179:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574432.631190:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574432.631201:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616426624/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574432.631208:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574432.631211:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574432.631213:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574432.631215:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6345 00000100:00080000:0.0:1788574432.631218:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574432.631221:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574432.631224:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574432.631231:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616426752/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574432.631246:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616426496:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788574432.631319:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574432.631357:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574432/real 1788574432] req@ffff9ada4557fb80 x1875449616426496/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574457 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574432.631375:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616426496/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574457 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574432.631385:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574432.631388:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574432.631391:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616426496:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574432.631396:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616426624:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574432.631415:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574432.631424:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574432/real 1788574432] req@ffff9ada4557e300 x1875449616426624/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574457 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574432.631433:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616426624/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574457 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574432.631439:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574432.631442:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574432.631444:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616426624:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574432.631468:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616426752:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574432.631478:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574432.631484:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574432/real 1788574432] req@ffff9ada4557dc00 x1875449616426752/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574457 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574432.631492:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616426752/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574457 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574432.631498:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574432.631500:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574432.631502:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616426752:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574432.822959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574433.143055:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574433.846972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574434.167109:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574434.870994:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574435.191031:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574435.894964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574436.215019:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574436.918921:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574437.239120:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574437.750978:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574437.750986:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574437.750989:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574437.750997:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6350 00000100:00080000:0.0:1788574437.751005:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574437.751011:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574437.751050:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616426880/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574437.751082:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574437.751087:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574437.751089:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574437.751093:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6350 00000100:00080000:0.0:1788574437.751097:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574437.751102:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574437.751113:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574437.751127:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616427008/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788574437.751138:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616426880:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788574437.751141:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574437.751145:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574437.751150:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574437.751156:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6350 00000100:00080000:0.0:1788574437.751160:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574437.751164:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574437.751169:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574437.751185:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616427136/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788574437.751212:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574437.751246:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574437/real 1788574437] req@ffff9ada4557d880 x1875449616426880/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574462 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574437.751265:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616426880/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574462 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574437.751276:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574437.751280:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574437.751283:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616426880:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574437.751303:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616427008:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574437.751321:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574437.751330:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574437/real 1788574437] req@ffff9ada4557f800 x1875449616427008/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574462 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574437.751338:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616427008/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574462 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574437.751344:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574437.751346:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574437.751348:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616427008:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574437.751352:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616427136:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574437.751362:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574437.751368:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574437/real 1788574437] req@ffff9ada4557ca80 x1875449616427136/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574462 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574437.751375:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616427136/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574462 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574437.751380:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574437.751382:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574437.751384:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616427136:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574437.942914:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574438.263047:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574438.966902:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574439.287036:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574439.991048:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574440.311065:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574441.014975:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574441.335063:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574442.038966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574442.359007:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574442.870961:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574442.870969:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574442.870972:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574442.870981:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6355 00000100:00080000:0.0:1788574442.870999:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574442.871005:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574442.871037:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616427264/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574442.871058:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574442.871061:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574442.871062:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574442.871065:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6355 00000100:00080000:0.0:1788574442.871067:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574442.871070:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574442.871083:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574442.871092:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616427392/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574442.871100:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1788574442.871104:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616427264:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788574442.871105:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574442.871107:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574442.871115:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6355 00000100:00080000:0.0:1788574442.871118:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574442.871120:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574442.871123:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574442.871132:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ce00 x1875449616427520/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788574442.871168:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574442.871199:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574442/real 1788574442] req@ffff9ada4557f480 x1875449616427264/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574467 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574442.871212:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616427264/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574467 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574442.871221:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574442.871224:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574442.871226:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616427264:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574442.871244:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616427392:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574442.871257:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574442.871265:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574442/real 1788574442] req@ffff9ada4557c700 x1875449616427392/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574467 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574442.871273:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616427392/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574467 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574442.871280:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574442.871283:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574442.871286:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616427392:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574442.871292:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616427520:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574442.871312:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574442.871324:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574442/real 1788574442] req@ffff9ada4557ce00 x1875449616427520/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574467 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574442.871337:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ce00 x1875449616427520/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574467 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574442.871345:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574442.871349:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574442.871352:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616427520:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574443.062959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574443.382995:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574444.086965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574444.407020:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574445.110965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574445.431081:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574446.134963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574446.454983:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574447.158951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574447.479013:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574447.990956:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574447.990964:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574447.990967:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574447.990975:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6360 00000100:00080000:0.0:1788574447.990984:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574447.990990:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574447.991020:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c000 x1875449616427648/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574447.991041:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574447.991044:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574447.991045:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574447.991047:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6360 00000100:00080000:0.0:1788574447.991050:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574447.991052:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574447.991060:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574447.991068:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c380 x1875449616427776/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574447.991076:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574447.991078:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574447.991080:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574447.991083:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6360 00000100:00080000:0.0:1788574447.991085:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574447.991087:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574447.991091:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574447.991093:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616427648:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574447.991103:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ce00 x1875449616427904/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788574447.991174:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616427776:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574447.991204:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616427904:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574448.182967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574448.503037:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574449.206916:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574449.527039:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574450.230951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574450.551017:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1788574451.063188:0:2370:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:2.0:1788574451.063227:0:2370:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1788574451.063242:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788574451.063284:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788574451.063295:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:1.0:1788574451.063339:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574447/real 1788574451] req@ffff9ada4557ce00 x1875449616427904/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574472 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574451.063363:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ce00 x1875449616427904/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574472 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574451.063373:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574451.063377:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574451.063380:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616427904:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574451.063386:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574447/real 1788574451] req@ffff9ada4557c000 x1875449616427648/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574472 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574451.063399:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c000 x1875449616427648/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574472 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574451.063407:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574451.063411:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574451.063415:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616427648:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574451.063422:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574447/real 1788574451] req@ffff9ada4557c380 x1875449616427776/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574472 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574451.063434:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c380 x1875449616427776/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574472 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574451.063441:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574451.063445:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574451.063448:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616427776:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574451.254926:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574451.575013:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574452.278957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574452.599067:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574453.111067:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574453.111077:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574453.111082:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574453.111093:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6365 00000100:00080000:2.0:1788574453.111103:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574453.111111:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574453.111148:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616428032/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574453.111187:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574453.111192:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574453.111194:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574453.111199:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6365 00000100:00080000:2.0:1788574453.111203:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574453.111208:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574453.111218:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574453.111233:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616428160/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1788574453.111242:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616428032:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788574453.111250:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574453.111255:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574453.111257:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574453.111262:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6365 00000100:00080000:2.0:1788574453.111267:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574453.111271:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574453.111277:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574453.111291:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616428288/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788574453.111319:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574453.111368:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574453/real 1788574453] req@ffff9ada60624380 x1875449616428032/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574478 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574453.111388:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616428032/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574478 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574453.111398:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574453.111401:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574453.111404:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616428032:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574453.111424:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616428160:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574453.111443:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574453.111456:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574453/real 1788574453] req@ffff9ada60626a00 x1875449616428160/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574478 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574453.111469:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616428160/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574478 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574453.111478:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574453.111481:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574453.111485:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616428160:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574453.111498:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616428288:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574453.111520:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574453.111532:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574453/real 1788574453] req@ffff9ada60624a80 x1875449616428288/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574478 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574453.111547:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616428288/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574478 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574453.111555:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574453.111558:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574453.111561:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616428288:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574453.302997:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574453.623017:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574454.326972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574454.647054:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574455.350998:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574455.671048:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574456.374938:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574456.695019:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574457.398953:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574457.719033:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574458.230958:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574458.230969:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574458.230973:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574458.230983:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6370 00000100:00080000:0.0:1788574458.230994:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574458.231001:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574458.231039:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616428416/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574458.231067:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574458.231071:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574458.231072:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574458.231075:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6370 00000100:00080000:0.0:1788574458.231078:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574458.231081:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574458.231089:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574458.231100:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616428544/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574458.231108:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574458.231111:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574458.231112:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574458.231118:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6370 00000100:00100000:1.0:1788574458.231118:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616428416:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788574458.231123:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574458.231127:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574458.231130:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574458.231140:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616428672/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788574458.231177:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574458.231207:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574458/real 1788574458] req@ffff9ada4557c700 x1875449616428416/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574483 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574458.231220:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616428416/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574483 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574458.231228:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574458.231231:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574458.231234:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616428416:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574458.231251:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616428544:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574458.231265:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574458.231272:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574458/real 1788574458] req@ffff9ada4557f480 x1875449616428544/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574483 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574458.231280:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616428544/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574483 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574458.231286:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574458.231287:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574458.231289:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616428544:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574458.231293:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616428672:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574458.231303:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574458.231309:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574458/real 1788574458] req@ffff9ada4557ca80 x1875449616428672/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574483 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574458.231316:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616428672/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574483 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574458.231322:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574458.231323:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574458.231325:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616428672:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574458.422947:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574458.743036:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574459.446998:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574459.767034:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574460.470962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574460.791041:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574461.494986:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574461.815029:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574462.518977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574462.839115:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574463.351025:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574463.351035:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574463.351039:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574463.351048:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6375 00000100:00080000:2.0:1788574463.351058:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574463.351064:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574463.351099:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616428800/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574463.351130:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574463.351140:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574463.351142:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574463.351146:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6375 00000100:00080000:2.0:1788574463.351150:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574463.351154:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574463.351164:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574463.351182:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616428800:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788574463.351183:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616428928/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574463.351201:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574463.351205:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574463.351208:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574463.351213:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6375 00000100:00080000:2.0:1788574463.351218:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574463.351223:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574463.351229:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574463.351244:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616429056/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788574463.351254:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574463.351296:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574463/real 1788574463] req@ffff9ada60625880 x1875449616428800/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574488 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574463.351311:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616428800/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574488 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574463.351319:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574463.351322:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574463.351324:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616428800:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574463.351345:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616428928:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574463.351362:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574463.351370:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574463/real 1788574463] req@ffff9ada60625500 x1875449616428928/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574488 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574463.351383:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616428928/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574488 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574463.351388:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574463.351390:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574463.351392:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616428928:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574463.351396:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616429056:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574463.351407:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574463.351413:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574463/real 1788574463] req@ffff9ada60625c00 x1875449616429056/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574488 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574463.351420:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616429056/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574488 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574463.351425:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574463.351427:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574463.351428:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616429056:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574463.542973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574463.863030:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574464.566964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574464.887073:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574465.591014:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574465.911054:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574466.614961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574466.935059:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574467.638980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574467.959058:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574468.471005:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574468.471013:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574468.471017:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574468.471025:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6381 00000100:00080000:2.0:1788574468.471033:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574468.471039:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574468.471070:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616429184/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574468.471094:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574468.471097:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574468.471098:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574468.471101:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6381 00000100:00080000:2.0:1788574468.471104:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574468.471106:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574468.471113:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574468.471120:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616429312/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574468.471128:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574468.471130:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574468.471132:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574468.471135:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6381 00000100:00080000:2.0:1788574468.471137:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574468.471139:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574468.471143:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574468.471146:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616429184:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788574468.471152:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616429440/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788574468.471202:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574468.471231:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574468/real 1788574468] req@ffff9ada60625f80 x1875449616429184/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574493 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574468.471249:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616429184/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574493 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574468.471257:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574468.471259:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574468.471262:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616429184:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574468.471267:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616429312:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574468.471282:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574468.471290:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574468/real 1788574468] req@ffff9ada60624e00 x1875449616429312/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574493 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574468.471298:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616429312/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574493 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574468.471303:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574468.471305:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574468.471308:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616429312:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574468.471325:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616429440:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574468.471335:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574468.471341:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574468/real 1788574468] req@ffff9ada60627480 x1875449616429440/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574493 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574468.471349:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616429440/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574493 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574468.471358:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574468.471360:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574468.471362:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616429440:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574468.662914:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574468.983040:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574469.686961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574470.007115:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788574470.711018:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574471.031011:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574471.734975:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574472.055035:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574472.758978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574473.079005:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788574473.591010:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574473.591036:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788574473.591041:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574473.591051:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6386 00000100:00080000:1.0:1788574473.591063:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574473.591071:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574473.591119:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ed80 x1875449616429568/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574473.591148:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574473.591153:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788574473.591155:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574473.591159:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6386 00000100:00080000:1.0:1788574473.591163:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574473.591167:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574473.591178:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574473.591193:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566dc00 x1875449616429696/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574473.591203:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616429568:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788574473.591213:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574473.591218:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788574473.591220:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574473.591225:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6386 00000100:00080000:1.0:1788574473.591230:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574473.591234:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574473.591240:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574473.591254:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f800 x1875449616429824/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574473.591265:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574473.591298:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574473/real 1788574473] req@ffff9ada4566ed80 x1875449616429568/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574498 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574473.591314:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ed80 x1875449616429568/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574498 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574473.591322:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574473.591325:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574473.591337:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616429568:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574473.591360:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616429696:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574473.591376:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574473.591384:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574473/real 1788574473] req@ffff9ada4566dc00 x1875449616429696/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574498 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574473.591393:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566dc00 x1875449616429696/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574498 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574473.591398:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574473.591401:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574473.591403:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616429696:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574473.591406:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616429824:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574473.591416:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574473.591421:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574473/real 1788574473] req@ffff9ada4566f800 x1875449616429824/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574498 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574473.591434:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f800 x1875449616429824/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574498 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574473.591438:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574473.591440:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574473.591442:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616429824:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574473.782948:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574474.103069:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574474.806985:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574475.126962:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574475.830998:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574476.150988:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574476.854933:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574477.175010:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574477.878915:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574478.199021:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788574478.710972:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574478.710979:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788574478.710983:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574478.710992:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6391 00000100:00080000:3.0:1788574478.711000:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574478.711019:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574478.711059:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2d80 x1875449616429952/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574478.711099:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574478.711103:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788574478.711106:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574478.711111:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6391 00000100:00080000:3.0:1788574478.711115:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574478.711120:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574478.711130:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574478.711133:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616429952:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574478.711143:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0a80 x1875449616430080/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574478.711154:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574478.711157:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788574478.711158:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574478.711161:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6391 00000100:00080000:3.0:1788574478.711164:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574478.711166:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574478.711169:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574478.711176:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2300 x1875449616430208/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788574478.711191:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574478.711226:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574478/real 1788574478] req@ffff9ada604e2d80 x1875449616429952/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574503 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574478.711247:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2d80 x1875449616429952/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574503 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574478.711258:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574478.711263:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574478.711267:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616429952:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574478.711286:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616430080:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574478.711305:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574478.711315:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574478/real 1788574478] req@ffff9ada604e0a80 x1875449616430080/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574503 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574478.711323:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0a80 x1875449616430080/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574503 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574478.711329:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574478.711331:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574478.711333:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616430080:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574478.711337:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616430208:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574478.711348:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574478.711358:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574478/real 1788574478] req@ffff9ada604e2300 x1875449616430208/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574503 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574478.711370:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2300 x1875449616430208/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574503 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574478.711380:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574478.711383:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574478.711386:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616430208:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574478.902989:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574479.223011:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574479.926974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574480.246981:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574480.950931:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574481.270986:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574481.974996:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574482.295027:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574482.998982:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574483.318967:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574483.830956:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574483.830965:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574483.830970:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574483.830981:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6396 00000100:00080000:0.0:1788574483.830992:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574483.831000:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574483.831041:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616430336/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574483.831071:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574483.831074:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574483.831076:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574483.831079:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6396 00000100:00080000:0.0:1788574483.831083:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574483.831086:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574483.831094:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574483.831107:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616430464/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574483.831115:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574483.831118:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574483.831119:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574483.831122:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6396 00000100:00080000:0.0:1788574483.831125:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1788574483.831149:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616430336:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788574483.831157:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574483.831163:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574483.831174:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616430592/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:1.0:1788574483.831218:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574483.831259:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574483/real 1788574483] req@ffff9ada4557f800 x1875449616430336/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574508 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574483.831280:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616430336/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574508 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574483.831291:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574483.831296:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574483.831300:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616430336:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574483.831308:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616430464:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574483.831333:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574483.831348:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574483/real 1788574483] req@ffff9ada4557d880 x1875449616430464/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574508 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574483.831362:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616430464/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574508 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574483.831371:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574483.831375:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574483.831379:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616430464:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574483.831403:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616430592:192.168.202.113@tcp:38:kworker.0 00000800:00000100:1.0:1788574483.831424:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574483.831437:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574483/real 1788574483] req@ffff9ada4557dc00 x1875449616430592/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574508 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574483.831450:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616430592/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574508 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574483.831465:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1788574483.831469:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:1.0:1788574483.831472:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616430592:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574484.022945:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574484.343207:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574485.047011:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574485.366999:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574486.070989:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788574486.390968:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574487.094991:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574487.415037:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574488.118940:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574488.439013:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788574488.950991:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574488.950999:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788574488.951002:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574488.951011:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6401 00000100:00080000:1.0:1788574488.951020:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574488.951025:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574488.951057:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ea00 x1875449616430720/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574488.951085:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574488.951087:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788574488.951089:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574488.951091:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6401 00000100:00080000:1.0:1788574488.951094:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574488.951096:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574488.951104:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574488.951113:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e300 x1875449616430848/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574488.951124:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574488.951128:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788574488.951129:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574488.951132:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6401 00000100:00080000:1.0:1788574488.951134:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574488.951136:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574488.951139:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574488.951146:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f480 x1875449616430976/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574488.951165:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616430720:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788574488.951266:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574488.951318:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574488/real 1788574488] req@ffff9ada4566ea00 x1875449616430720/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574513 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574488.951350:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ea00 x1875449616430720/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574513 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574488.951360:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574488.951364:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574488.951368:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616430720:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574488.951374:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616430848:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574488.951406:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574488.951417:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574488/real 1788574488] req@ffff9ada4566e300 x1875449616430848/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574513 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574488.951426:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e300 x1875449616430848/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574513 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574488.951433:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574488.951435:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574488.951438:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616430848:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574488.951461:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616430976:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574488.951474:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574488.951481:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574488/real 1788574488] req@ffff9ada4566f480 x1875449616430976/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574513 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574488.951494:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f480 x1875449616430976/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574513 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574488.951500:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574488.951502:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574488.951504:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616430976:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574489.142966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574489.463070:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574490.166943:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574490.487000:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574491.190958:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574491.511088:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574492.214966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574492.535057:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574493.238930:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574493.559064:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574494.071010:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574494.071018:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574494.071022:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574494.071031:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6406 00000100:00080000:0.0:1788574494.071040:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574494.071046:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574494.071088:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616431104/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574494.071109:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574494.071111:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574494.071113:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574494.071115:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6406 00000100:00080000:0.0:1788574494.071118:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574494.071120:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574494.071127:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574494.071146:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616431232/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574494.071154:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574494.071164:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1788574494.071164:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616431104:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788574494.071168:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574494.071176:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6406 00000100:00080000:0.0:1788574494.071179:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574494.071181:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574494.071184:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574494.071193:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616431360/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574494.071258:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574494.071312:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574494/real 1788574494] req@ffff9ada4557e300 x1875449616431104/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574519 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574494.071327:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616431104/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574519 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574494.071335:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574494.071338:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574494.071341:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616431104:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574494.071359:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616431232:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574494.071378:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574494.071387:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574494/real 1788574494] req@ffff9ada4557fb80 x1875449616431232/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574519 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574494.071395:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616431232/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574519 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574494.071400:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574494.071403:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574494.071405:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616431232:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574494.071408:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616431360:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574494.071419:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574494.071424:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574494/real 1788574494] req@ffff9ada4557ed80 x1875449616431360/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574519 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574494.071431:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616431360/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574519 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574494.071436:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574494.071438:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574494.071440:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616431360:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574494.262960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574494.583055:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574495.286928:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574495.607106:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574496.310977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574496.631068:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574497.334965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574497.655088:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574498.358971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574498.679095:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574499.190950:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574499.190958:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574499.190962:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574499.190970:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6411 00000100:00080000:0.0:1788574499.190979:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574499.190985:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574499.191021:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616431488/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574499.191053:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574499.191057:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574499.191058:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574499.191061:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6411 00000100:00080000:0.0:1788574499.191065:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574499.191068:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574499.191075:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574499.191087:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616431616/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574499.191100:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616431488:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788574499.191112:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574499.191116:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574499.191119:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574499.191124:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6411 00000100:00080000:0.0:1788574499.191130:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574499.191134:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574499.191147:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1788574499.191159:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574499.191160:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d500 x1875449616431744/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574499.191193:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574499/real 1788574499] req@ffff9ada4557ea00 x1875449616431488/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574524 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574499.191208:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616431488/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574524 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574499.191216:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574499.191218:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574499.191221:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616431488:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574499.191238:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616431616:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574499.191269:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574499.191286:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574499/real 1788574499] req@ffff9ada4557e680 x1875449616431616/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574524 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574499.191295:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616431616/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574524 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574499.191304:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574499.191306:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574499.191309:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616431616:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574499.191313:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616431744:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574499.191327:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574499.191333:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574499/real 1788574499] req@ffff9ada4557d500 x1875449616431744/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574524 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574499.191341:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d500 x1875449616431744/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574524 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574499.191347:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574499.191348:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574499.191350:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616431744:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574499.383004:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574499.703115:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574500.406939:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574500.727012:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574501.430968:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574501.751032:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574502.454948:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574502.775021:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574503.479009:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574503.799057:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788574504.310964:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574504.310973:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788574504.310977:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574504.310985:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6416 00000100:00080000:3.0:1788574504.310994:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574504.311000:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574504.311034:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0000 x1875449616431872/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574504.311061:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574504.311064:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788574504.311065:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574504.311068:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6416 00000100:00080000:3.0:1788574504.311071:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574504.311074:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574504.311083:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574504.311096:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616431872:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574504.311096:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1c00 x1875449616432000/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574504.311110:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574504.311112:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788574504.311114:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574504.311117:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6416 00000100:00080000:3.0:1788574504.311119:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574504.311122:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574504.311124:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574504.311133:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3b80 x1875449616432128/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788574504.311158:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574504.311185:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574504/real 1788574504] req@ffff9ada604e0000 x1875449616431872/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574529 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574504.311198:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0000 x1875449616431872/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574529 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574504.311205:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574504.311208:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574504.311211:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616431872:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574504.311228:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616432000:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574504.311243:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574504.311256:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574504/real 1788574504] req@ffff9ada604e1c00 x1875449616432000/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574529 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574504.311270:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1c00 x1875449616432000/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574529 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574504.311279:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574504.311307:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574504.311311:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616432000:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574504.311316:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616432128:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574504.311334:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574504.311342:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574504/real 1788574504] req@ffff9ada604e3b80 x1875449616432128/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574529 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574504.311350:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3b80 x1875449616432128/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574529 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574504.311356:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574504.311358:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574504.311360:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616432128:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574504.502945:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574504.822977:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574505.526961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574505.846988:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574506.550978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574506.871055:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574507.574957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574507.895045:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574508.598952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574508.919073:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788574509.430967:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574509.430980:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788574509.430984:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574509.430996:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6421 00000100:00080000:3.0:1788574509.431007:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574509.431015:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574509.431065:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3100 x1875449616432256/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574509.431093:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574509.431096:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788574509.431097:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574509.431100:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6421 00000100:00080000:3.0:1788574509.431103:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574509.431105:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574509.431115:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574509.431125:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1f80 x1875449616432384/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574509.431132:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574509.431136:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788574509.431137:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574509.431140:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6421 00000100:00080000:3.0:1788574509.431142:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574509.431144:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574509.431147:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574509.431151:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616432256:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574509.431157:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2680 x1875449616432512/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788574509.431234:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574509.431279:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574509/real 1788574509] req@ffff9ada604e3100 x1875449616432256/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574534 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574509.431301:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3100 x1875449616432256/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574534 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574509.431347:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574509.431351:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574509.431354:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616432256:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574509.431361:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616432384:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574509.431388:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574509.431399:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574509/real 1788574509] req@ffff9ada604e1f80 x1875449616432384/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574534 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574509.431408:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1f80 x1875449616432384/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574534 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574509.431414:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574509.431416:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574509.431419:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616432384:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574509.431442:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616432512:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574509.431462:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574509.431469:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574509/real 1788574509] req@ffff9ada604e2680 x1875449616432512/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574534 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574509.431477:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2680 x1875449616432512/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574534 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574509.431483:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574509.431485:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574509.431487:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616432512:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574509.623004:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574509.943018:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574510.646974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574510.966987:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574511.671061:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574511.990967:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574512.694953:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574513.015030:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574513.718980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574514.039067:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574514.551011:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574514.551019:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574514.551023:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574514.551031:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6427 00000100:00080000:0.0:1788574514.551040:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574514.551046:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574514.551083:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557df80 x1875449616432640/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574514.551106:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574514.551109:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574514.551110:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574514.551113:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6427 00000100:00080000:0.0:1788574514.551115:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574514.551118:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574514.551126:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574514.551135:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d180 x1875449616432768/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574514.551144:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616432640:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788574514.551145:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574514.551152:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574514.551155:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574514.551158:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6427 00000100:00080000:0.0:1788574514.551161:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574514.551163:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574514.551167:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574514.551176:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d500 x1875449616432896/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574514.551231:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616432768:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574514.551253:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616432896:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574514.742943:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574515.063043:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574515.766972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574516.087029:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574516.790961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574517.111021:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1788574517.623120:0:2372:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:0.0:1788574517.623156:0:2372:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1788574517.623169:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:0.0:1788574517.623206:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:0.0:1788574517.623216:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:3.0:1788574517.623277:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574514/real 1788574517] req@ffff9ada4557d180 x1875449616432768/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574539 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574517.623310:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d180 x1875449616432768/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574539 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574517.623323:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574517.623328:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574517.623332:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616432768:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574517.623341:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574514/real 1788574517] req@ffff9ada4557d500 x1875449616432896/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574539 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574517.623354:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d500 x1875449616432896/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574539 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574517.623378:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574517.623382:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574517.623385:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616432896:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574517.623392:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574514/real 1788574517] req@ffff9ada4557df80 x1875449616432640/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574539 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574517.623406:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557df80 x1875449616432640/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574539 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574517.623414:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574517.623418:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574517.623421:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616432640:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574517.814992:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574518.134990:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574518.838959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574519.159022:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788574519.671001:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574519.671013:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788574519.671018:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574519.671029:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6432 00000100:00080000:1.0:1788574519.671039:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574519.671047:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574519.671083:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d500 x1875449616433024/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574519.671120:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574519.671124:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788574519.671127:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574519.671131:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6432 00000100:00080000:1.0:1788574519.671135:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574519.671140:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574519.671150:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574519.671166:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616433024:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574519.671167:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ca80 x1875449616433152/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574519.671184:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574519.671188:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788574519.671190:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574519.671195:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6432 00000100:00080000:1.0:1788574519.671199:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574519.671205:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574519.671210:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574519.671224:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d180 x1875449616433280/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574519.671227:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574519.671266:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574519/real 1788574519] req@ffff9ada4566d500 x1875449616433024/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574544 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574519.671288:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d500 x1875449616433024/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574544 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574519.671305:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574519.671310:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574519.671314:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616433024:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574519.671356:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616433152:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574519.671397:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574519.671411:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574519/real 1788574519] req@ffff9ada4566ca80 x1875449616433152/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574544 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574519.671425:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ca80 x1875449616433152/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574544 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574519.671433:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574519.671436:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574519.671439:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616433152:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574519.671445:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616433280:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574519.671464:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574519.671475:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574519/real 1788574519] req@ffff9ada4566d180 x1875449616433280/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574544 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574519.671487:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d180 x1875449616433280/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574544 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574519.671495:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574519.671498:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574519.671501:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616433280:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574519.862963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574520.183040:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574520.886957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574521.207065:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574521.910960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574522.231018:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574522.934929:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574523.255001:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574523.958959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574524.279005:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788574524.791036:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574524.791046:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788574524.791049:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574524.791058:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6437 00000100:00080000:3.0:1788574524.791071:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574524.791077:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574524.791110:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2a00 x1875449616433408/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574524.791134:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574524.791137:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788574524.791139:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574524.791142:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6437 00000100:00080000:3.0:1788574524.791144:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574524.791146:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574524.791153:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574524.791163:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1500 x1875449616433536/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574524.791172:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1788574524.791176:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616433408:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788574524.791177:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788574524.791180:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574524.791188:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6437 00000100:00080000:3.0:1788574524.791190:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574524.791192:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574524.791195:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574524.791205:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0e00 x1875449616433664/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788574524.791241:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574524.791270:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574524/real 1788574524] req@ffff9ada604e2a00 x1875449616433408/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574549 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574524.791284:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2a00 x1875449616433408/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574549 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574524.791292:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574524.791295:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574524.791298:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616433408:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574524.791316:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616433536:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574524.791330:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574524.791338:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574524/real 1788574524] req@ffff9ada604e1500 x1875449616433536/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574549 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574524.791350:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1500 x1875449616433536/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574549 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574524.791356:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574524.791358:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574524.791360:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616433536:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574524.791364:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616433664:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574524.791375:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574524.791381:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574524/real 1788574524] req@ffff9ada604e0e00 x1875449616433664/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574549 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574524.791389:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0e00 x1875449616433664/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574549 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574524.791394:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574524.791395:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574524.791397:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616433664:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574524.982961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574525.303021:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574526.007060:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574526.327033:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574527.030947:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574527.351005:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574528.054952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574528.375054:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574529.078934:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574529.399063:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788574529.910985:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574529.910997:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788574529.911001:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574529.911012:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6442 00000100:00080000:3.0:1788574529.911024:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574529.911032:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574529.911080:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0e00 x1875449616433792/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574529.911109:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574529.911112:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788574529.911114:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574529.911117:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6442 00000100:00080000:3.0:1788574529.911120:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574529.911122:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574529.911132:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574529.911141:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1500 x1875449616433920/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574529.911149:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574529.911152:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788574529.911153:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574529.911156:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6442 00000100:00080000:3.0:1788574529.911158:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574529.911160:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574529.911164:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574529.911170:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2a00 x1875449616434048/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788574529.911206:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616433792:192.168.202.113@tcp:250:kworker.0 00000800:00000100:0.0:1788574529.911271:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574529.911304:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574529/real 1788574529] req@ffff9ada604e0e00 x1875449616433792/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574554 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574529.911319:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0e00 x1875449616433792/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574554 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574529.911329:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574529.911332:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574529.911335:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616433792:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574529.911340:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616433920:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574529.911355:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574529.911363:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574529/real 1788574529] req@ffff9ada604e1500 x1875449616433920/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574554 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574529.911375:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1500 x1875449616433920/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574554 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574529.911384:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574529.911388:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574529.911391:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616433920:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574529.911399:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616434048:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574529.911421:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574529.911449:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574529/real 1788574529] req@ffff9ada604e2a00 x1875449616434048/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574554 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574529.911463:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2a00 x1875449616434048/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574554 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574529.911471:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574529.911475:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574529.911478:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616434048:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574530.103016:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574530.423046:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574531.126973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574531.446970:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574532.150964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574532.471009:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574533.174973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574533.495001:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574534.198981:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574534.519172:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788574535.030992:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574535.031000:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788574535.031003:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574535.031012:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6447 00000100:00080000:1.0:1788574535.031021:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574535.031028:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574535.031060:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f100 x1875449616434176/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574535.031084:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574535.031086:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788574535.031088:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574535.031090:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6447 00000100:00080000:1.0:1788574535.031093:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574535.031095:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574535.031103:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574535.031111:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c380 x1875449616434304/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574535.031121:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574535.031125:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788574535.031128:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1788574535.031130:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616434176:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788574535.031136:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6447 00000100:00080000:1.0:1788574535.031142:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574535.031146:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574535.031152:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574535.031168:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c700 x1875449616434432/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788574535.031191:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574535.031220:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574535/real 1788574535] req@ffff9ada4566f100 x1875449616434176/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574560 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574535.031234:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f100 x1875449616434176/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574560 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574535.031242:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574535.031245:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574535.031248:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616434176:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574535.031265:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616434304:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574535.031280:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574535.031288:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574535/real 1788574535] req@ffff9ada4566c380 x1875449616434304/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574560 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574535.031296:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c380 x1875449616434304/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574560 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574535.031302:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574535.031304:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574535.031306:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616434304:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574535.031310:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616434432:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574535.031322:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574535.031328:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574535/real 1788574535] req@ffff9ada4566c700 x1875449616434432/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574560 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574535.031335:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c700 x1875449616434432/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574560 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574535.031340:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574535.031342:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574535.031344:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616434432:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574535.222983:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574535.543040:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574536.246938:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574536.567107:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574537.270953:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574537.591155:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574538.294981:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574538.615083:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574539.318988:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574539.639024:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574540.150980:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574540.150992:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574540.150997:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574540.151008:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6452 00000100:00080000:0.0:1788574540.151020:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574540.151036:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574540.151082:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616434560/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574540.151109:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574540.151112:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574540.151114:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574540.151117:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6452 00000100:00080000:0.0:1788574540.151119:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574540.151122:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574540.151133:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574540.151143:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616434688/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574540.151152:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574540.151157:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1788574540.151158:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616434560:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788574540.151159:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574540.151168:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6452 00000100:00080000:0.0:1788574540.151171:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574540.151173:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574540.151177:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574540.151187:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616434816/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574540.151234:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574540.151269:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574540/real 1788574540] req@ffff9ada4557e680 x1875449616434560/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574565 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574540.151286:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616434560/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574565 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574540.151297:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574540.151300:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574540.151303:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616434560:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574540.151326:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616434688:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574540.151342:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574540.151351:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574540/real 1788574540] req@ffff9ada4557ea00 x1875449616434688/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574565 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574540.151359:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616434688/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574565 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574540.151365:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574540.151367:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574540.151369:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616434688:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574540.151373:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616434816:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574540.151396:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574540.151403:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574540/real 1788574540] req@ffff9ada4557ed80 x1875449616434816/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574565 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574540.151411:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616434816/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574565 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574540.151417:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574540.151419:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574540.151421:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616434816:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574540.342988:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574540.663066:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574541.367020:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574541.687156:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574542.390928:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574542.711001:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574543.414908:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574543.735026:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574544.438982:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574544.759011:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788574545.271039:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574545.271050:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788574545.271054:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574545.271064:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6457 00000100:00080000:3.0:1788574545.271076:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574545.271084:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574545.271122:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2680 x1875449616434944/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574545.271157:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574545.271161:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788574545.271164:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574545.271169:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6457 00000100:00080000:3.0:1788574545.271173:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574545.271178:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574545.271188:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574545.271203:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1f80 x1875449616435072/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788574545.271219:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616434944:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788574545.271221:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574545.271233:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788574545.271239:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574545.271244:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6457 00000100:00080000:3.0:1788574545.271248:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574545.271252:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574545.271259:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574545.271274:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3100 x1875449616435200/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788574545.271336:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574545.271384:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574545/real 1788574545] req@ffff9ada604e2680 x1875449616434944/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574570 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574545.271407:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2680 x1875449616434944/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574570 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574545.271420:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574545.271425:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574545.271429:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616434944:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574545.271460:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616435072:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574545.271484:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574545.271497:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574545/real 1788574545] req@ffff9ada604e1f80 x1875449616435072/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574570 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574545.271510:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1f80 x1875449616435072/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574570 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574545.271533:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574545.271537:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574545.271541:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616435072:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574545.271548:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616435200:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574545.271568:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574545.271580:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574545/real 1788574545] req@ffff9ada604e3100 x1875449616435200/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574570 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574545.271592:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3100 x1875449616435200/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574570 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574545.271600:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574545.271604:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574545.271608:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616435200:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574545.462964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574545.783019:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574546.486920:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574546.806996:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574547.510931:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574547.831014:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574548.534938:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574548.855031:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574549.558946:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574549.879095:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574550.390933:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574550.390940:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574550.390944:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574550.390952:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6462 00000100:00080000:0.0:1788574550.390960:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574550.390966:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574550.390998:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616435328/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574550.391021:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574550.391023:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574550.391025:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574550.391028:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6462 00000100:00080000:0.0:1788574550.391030:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574550.391034:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574550.391041:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574550.391050:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616435328:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574550.391051:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616435456/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574550.391062:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574550.391065:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574550.391066:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574550.391069:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6462 00000100:00080000:0.0:1788574550.391071:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574550.391073:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574550.391076:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574550.391083:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616435584/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574550.391123:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574550.391164:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574550/real 1788574550] req@ffff9ada4557fb80 x1875449616435328/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574575 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574550.391180:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616435328/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574575 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574550.391188:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574550.391191:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574550.391194:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616435328:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574550.391212:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616435456:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574550.391230:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574550.391239:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574550/real 1788574550] req@ffff9ada4557e300 x1875449616435456/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574575 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574550.391247:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616435456/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574575 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574550.391252:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574550.391255:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574550.391257:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616435456:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574550.391260:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616435584:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574550.391272:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574550.391277:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574550/real 1788574550] req@ffff9ada4557dc00 x1875449616435584/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574575 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574550.391289:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616435584/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574575 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574550.391294:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574550.391296:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574550.391298:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616435584:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574550.582973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574550.903053:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574551.606930:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574551.927083:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574552.630971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574552.951001:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574553.654938:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574553.975012:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574554.678974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574554.999095:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574555.510976:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574555.510986:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574555.510990:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574555.511000:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6468 00000100:00080000:0.0:1788574555.511011:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574555.511018:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574555.511058:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616435712/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574555.511088:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574555.511092:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574555.511094:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574555.511098:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6468 00000100:00080000:0.0:1788574555.511102:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574555.511106:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574555.511115:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574555.511128:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616435840/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574555.511138:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616435712:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788574555.511142:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574555.511147:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574555.511149:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574555.511154:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6468 00000100:00080000:0.0:1788574555.511159:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574555.511162:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574555.511167:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574555.511179:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616435968/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574555.511199:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574555.511231:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574555/real 1788574555] req@ffff9ada4557d880 x1875449616435712/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574580 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574555.511246:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616435712/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574580 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574555.511254:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574555.511256:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574555.511259:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616435712:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574555.511277:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616435840:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574555.511292:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574555.511300:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574555/real 1788574555] req@ffff9ada4557f800 x1875449616435840/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574580 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574555.511308:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616435840/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574580 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574555.511313:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574555.511315:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574555.511317:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616435840:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574555.511320:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616435968:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574555.511330:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574555.511336:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574555/real 1788574555] req@ffff9ada4557ca80 x1875449616435968/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574580 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574555.511343:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616435968/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574580 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574555.511348:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574555.511349:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574555.511351:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616435968:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574555.702916:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574556.023050:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574556.726964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574557.046974:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574557.750998:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574558.071057:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574558.774962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574559.095029:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574559.798957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574560.119025:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788574560.631007:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574560.631015:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788574560.631018:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574560.631027:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6473 00000100:00080000:1.0:1788574560.631036:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574560.631042:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574560.631077:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ce00 x1875449616436096/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574560.631100:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574560.631103:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788574560.631105:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574560.631107:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6473 00000100:00080000:1.0:1788574560.631110:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574560.631112:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574560.631120:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574560.631128:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d880 x1875449616436224/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574560.631135:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574560.631138:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788574560.631140:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574560.631143:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6473 00000100:00080000:1.0:1788574560.631145:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574560.631151:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574560.631151:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616436096:192.168.202.113@tcp:250:kworker.0 00000008:00100000:1.0:1788574560.631158:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574560.631165:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e680 x1875449616436352/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574560.631216:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574560.631252:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574560/real 1788574560] req@ffff9ada4566ce00 x1875449616436096/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574585 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574560.631281:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ce00 x1875449616436096/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574585 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574560.631293:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574560.631297:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574560.631300:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616436096:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574560.631306:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616436224:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574560.631326:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574560.631336:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574560/real 1788574560] req@ffff9ada4566d880 x1875449616436224/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574585 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574560.631344:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d880 x1875449616436224/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574585 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574560.631350:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574560.631353:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574560.631355:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616436224:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574560.631374:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616436352:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574560.631386:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574560.631392:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574560/real 1788574560] req@ffff9ada4566e680 x1875449616436352/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574585 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574560.631400:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e680 x1875449616436352/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574585 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574560.631406:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574560.631407:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574560.631409:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616436352:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574560.822962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574561.142987:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574561.846962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574562.167012:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574562.870952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574563.191076:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574563.894974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574564.215032:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574564.918944:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574565.238988:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788574565.750988:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574565.750996:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788574565.751000:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574565.751008:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6478 00000100:00080000:3.0:1788574565.751018:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574565.751025:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574565.751064:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3b80 x1875449616436480/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574565.751092:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574565.751095:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788574565.751096:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574565.751099:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6478 00000100:00080000:3.0:1788574565.751102:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574565.751105:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574565.751114:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574565.751124:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1c00 x1875449616436608/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574565.751137:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1788574565.751141:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616436480:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788574565.751141:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788574565.751144:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574565.751150:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6478 00000100:00080000:3.0:1788574565.751153:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574565.751155:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574565.751159:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574565.751168:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0000 x1875449616436736/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788574565.751215:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574565.751249:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574565/real 1788574565] req@ffff9ada604e3b80 x1875449616436480/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574590 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574565.751268:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3b80 x1875449616436480/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574590 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574565.751280:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574565.751284:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574565.751288:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616436480:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574565.751312:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616436608:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574565.751330:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574565.751340:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574565/real 1788574565] req@ffff9ada604e1c00 x1875449616436608/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574590 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574565.751348:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1c00 x1875449616436608/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574590 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574565.751354:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574565.751356:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574565.751358:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616436608:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574565.751362:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616436736:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574565.751372:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574565.751378:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574565/real 1788574565] req@ffff9ada604e0000 x1875449616436736/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574590 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574565.751385:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0000 x1875449616436736/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574590 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574565.751390:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574565.751392:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574565.751394:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616436736:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574565.942940:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574566.263023:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574566.966945:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574567.287051:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574567.990951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574568.311015:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574569.014969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574569.335041:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574570.038970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574570.358999:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574570.871001:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574570.871014:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574570.871026:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574570.871038:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6483 00000100:00080000:0.0:1788574570.871049:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574570.871057:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574570.871097:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616436864/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574570.871128:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574570.871133:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574570.871135:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574570.871140:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6483 00000100:00080000:0.0:1788574570.871145:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574570.871151:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574570.871162:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574570.871168:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616436864:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574570.871178:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616436992/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574570.871193:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574570.871197:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574570.871199:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574570.871204:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6483 00000100:00080000:0.0:1788574570.871208:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574570.871213:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574570.871218:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574570.871232:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c380 x1875449616437120/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574570.871236:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574570.871285:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574570/real 1788574570] req@ffff9ada4557f480 x1875449616436864/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574595 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574570.871306:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616436864/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574595 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574570.871317:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574570.871322:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574570.871326:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616436864:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574570.871347:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616436992:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574570.871369:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574570.871381:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574570/real 1788574570] req@ffff9ada4557c700 x1875449616436992/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574595 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574570.871394:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616436992/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574595 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574570.871403:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574570.871407:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574570.871410:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616436992:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574570.871416:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616437120:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574570.871433:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574570.871442:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574570/real 1788574570] req@ffff9ada4557c380 x1875449616437120/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574595 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574570.871454:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c380 x1875449616437120/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574595 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574570.871462:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574570.871465:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574570.871469:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616437120:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574571.062959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574571.383020:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574572.086993:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574572.407042:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574573.110929:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574573.431000:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574574.135009:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574574.455067:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574575.158913:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574575.479073:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788574575.990986:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574575.990996:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788574575.991000:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574575.991011:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6488 00000100:00080000:1.0:1788574575.991022:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574575.991030:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574575.991071:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e680 x1875449616437248/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574575.991098:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574575.991102:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788574575.991104:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574575.991109:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6488 00000100:00080000:1.0:1788574575.991113:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574575.991118:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574575.991129:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574575.991141:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616437248:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574575.991146:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d880 x1875449616437376/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574575.991161:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574575.991164:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788574575.991166:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574575.991171:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6488 00000100:00080000:1.0:1788574575.991175:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574575.991180:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574575.991185:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1788574575.991199:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788574575.991201:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ce00 x1875449616437504/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574575.991240:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574575/real 1788574575] req@ffff9ada4566e680 x1875449616437248/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574600 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574575.991256:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e680 x1875449616437248/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574600 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574575.991264:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574575.991267:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574575.991270:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616437248:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574575.991288:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616437376:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574575.991303:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574575.991312:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574575/real 1788574575] req@ffff9ada4566d880 x1875449616437376/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574600 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574575.991320:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d880 x1875449616437376/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574600 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574575.991326:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574575.991328:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574575.991330:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616437376:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574575.991334:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616437504:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574575.991345:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574575.991351:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574575/real 1788574575] req@ffff9ada4566ce00 x1875449616437504/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574600 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574575.991358:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ce00 x1875449616437504/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574600 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574575.991363:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574575.991365:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574575.991367:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616437504:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574576.182999:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574576.503035:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574577.207024:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574577.527179:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574578.230981:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574578.551046:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574579.254982:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574579.575097:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574580.279001:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574580.599003:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574581.110981:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574581.110992:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574581.111004:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574581.111014:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6493 00000100:00080000:0.0:1788574581.111023:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574581.111029:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574581.111067:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c000 x1875449616437632/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574581.111096:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574581.111100:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574581.111101:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574581.111104:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6493 00000100:00080000:0.0:1788574581.111107:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574581.111110:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574581.111118:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574581.111129:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ce00 x1875449616437760/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574581.111136:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574581.111138:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574581.111140:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574581.111142:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6493 00000100:00080000:0.0:1788574581.111144:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574581.111147:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574581.111151:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574581.111159:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c380 x1875449616437888/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574581.111172:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616437632:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574581.111252:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616437760:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574581.111279:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616437888:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574581.303003:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574581.623091:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574582.326969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574582.646977:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574583.350977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574583.671036:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1788574584.183153:0:2373:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:2.0:1788574584.183196:0:2373:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1788574584.183213:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788574584.183254:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788574584.183266:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:3.0:1788574584.183315:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574581/real 1788574584] req@ffff9ada4557c380 x1875449616437888/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574606 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574584.183345:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c380 x1875449616437888/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574606 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574584.183356:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574584.183361:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574584.183366:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616437888:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574584.183374:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574581/real 1788574584] req@ffff9ada4557c000 x1875449616437632/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574606 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574584.183389:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c000 x1875449616437632/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574606 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574584.183399:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574584.183416:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574584.183421:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616437632:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574584.183428:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574581/real 1788574584] req@ffff9ada4557ce00 x1875449616437760/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574606 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574584.183442:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ce00 x1875449616437760/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574606 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574584.183449:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574584.183453:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574584.183456:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616437760:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574584.374966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574584.695065:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574585.398915:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574585.718984:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788574586.230998:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574586.231007:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788574586.231011:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574586.231019:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6498 00000100:00080000:1.0:1788574586.231028:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574586.231035:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574586.231065:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c700 x1875449616438016/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574586.231087:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574586.231090:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788574586.231092:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574586.231095:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6498 00000100:00080000:1.0:1788574586.231097:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574586.231099:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574586.231108:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574586.231117:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c380 x1875449616438144/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574586.231124:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574586.231127:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788574586.231128:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574586.231130:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6498 00000100:00080000:1.0:1788574586.231133:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574586.231138:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574586.231138:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616438016:192.168.202.113@tcp:250:kworker.0 00000008:00100000:1.0:1788574586.231145:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574586.231153:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f100 x1875449616438272/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574586.231203:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574586.231234:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574586/real 1788574586] req@ffff9ada4566c700 x1875449616438016/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574611 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574586.231254:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c700 x1875449616438016/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574611 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574586.231265:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574586.231269:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574586.231273:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616438016:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574586.231280:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616438144:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574586.231310:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574586.231321:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574586/real 1788574586] req@ffff9ada4566c380 x1875449616438144/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574611 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574586.231334:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c380 x1875449616438144/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574611 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574586.231343:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574586.231347:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574586.231350:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616438144:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574586.231376:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616438272:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574586.231406:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574586.231419:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574586/real 1788574586] req@ffff9ada4566f100 x1875449616438272/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574611 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574586.231433:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f100 x1875449616438272/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574611 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574586.231452:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574586.231456:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574586.231460:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616438272:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574586.422965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574586.743051:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574587.446934:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574587.766996:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574588.470951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574588.791065:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574589.494969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574589.815041:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574590.518955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574590.839009:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574591.350982:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574591.350991:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574591.350994:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574591.351002:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6503 00000100:00080000:0.0:1788574591.351011:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574591.351018:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574591.351051:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616438400/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574591.351075:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574591.351078:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574591.351079:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574591.351082:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6503 00000100:00080000:0.0:1788574591.351084:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574591.351086:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574591.351094:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574591.351102:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616438528/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574591.351108:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574591.351115:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574591.351117:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574591.351120:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6503 00000100:00080000:0.0:1788574591.351122:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574591.351124:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574591.351126:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574591.351132:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616438656/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574591.351165:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616438400:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788574591.351219:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574591.351248:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574591/real 1788574591] req@ffff9ada4557c700 x1875449616438400/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574616 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574591.351263:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616438400/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574616 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574591.351271:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574591.351275:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574591.351278:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616438400:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574591.351283:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616438528:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574591.351296:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574591.351303:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574591/real 1788574591] req@ffff9ada4557f480 x1875449616438528/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574616 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574591.351311:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616438528/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574616 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574591.351316:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574591.351318:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574591.351320:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616438528:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574591.351323:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616438656:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574591.351333:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574591.351339:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574591/real 1788574591] req@ffff9ada4557ca80 x1875449616438656/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574616 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574591.351346:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616438656/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574616 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574591.351351:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574591.351353:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574591.351355:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616438656:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574591.542974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574591.863479:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574592.566937:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574592.887038:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574593.590968:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574593.911043:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574594.614934:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574594.935039:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574595.638957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574595.959010:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574596.470957:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574596.470966:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574596.470969:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574596.470977:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6509 00000100:00080000:2.0:1788574596.470986:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574596.470993:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574596.471024:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616438784/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574596.471046:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574596.471049:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574596.471051:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574596.471053:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6509 00000100:00080000:2.0:1788574596.471056:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574596.471059:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574596.471066:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574596.471075:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616438912/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574596.471083:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574596.471087:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574596.471091:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1788574596.471092:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616438784:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788574596.471098:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6509 00000100:00080000:2.0:1788574596.471104:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574596.471108:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574596.471114:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574596.471129:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616439040/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574596.471158:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574596.471193:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574596/real 1788574596] req@ffff9ada60625180 x1875449616438784/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574621 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574596.471212:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616438784/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574621 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574596.471221:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574596.471225:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574596.471229:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616438784:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574596.471251:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616438912:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574596.471273:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574596.471285:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574596/real 1788574596] req@ffff9ada60626d80 x1875449616438912/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574621 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574596.471298:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616438912/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574621 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574596.471306:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574596.471310:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574596.471314:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616438912:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574596.471320:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616439040:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574596.471354:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574596.471366:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574596/real 1788574596] req@ffff9ada60627b80 x1875449616439040/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574621 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574596.471378:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616439040/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574621 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574596.471386:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574596.471390:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574596.471393:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616439040:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574596.662959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574596.982990:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574597.686963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574598.007041:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574598.710972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574599.031042:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574599.734960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574600.054983:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574600.758944:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574601.079050:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574601.590995:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574601.591007:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574601.591011:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574601.591022:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6514 00000100:00080000:0.0:1788574601.591034:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574601.591042:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574601.591081:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616439168/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574601.591113:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574601.591117:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574601.591119:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574601.591123:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6514 00000100:00080000:0.0:1788574601.591127:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574601.591132:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574601.591141:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574601.591154:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616439296/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574601.591165:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574601.591169:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574601.591171:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574601.591174:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6514 00000100:00080000:0.0:1788574601.591178:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574601.591181:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574601.591187:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616439168:192.168.202.113@tcp:250:kworker.0 00000008:00100000:0.0:1788574601.591189:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574601.591205:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616439424/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574601.591254:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574601.591283:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574601/real 1788574601] req@ffff9ada4557f800 x1875449616439168/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574626 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574601.591298:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616439168/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574626 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574601.591329:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574601.591332:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574601.591335:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616439168:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574601.591340:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616439296:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574601.591355:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574601.591363:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574601/real 1788574601] req@ffff9ada4557d880 x1875449616439296/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574626 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574601.591371:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616439296/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574626 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574601.591376:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574601.591378:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574601.591380:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616439296:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574601.591399:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616439424:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574601.591409:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574601.591414:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574601/real 1788574601] req@ffff9ada4557dc00 x1875449616439424/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574626 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574601.591421:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616439424/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574626 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574601.591426:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574601.591428:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574601.591430:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616439424:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574601.782951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574602.102995:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574602.806951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574603.127081:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574603.830967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574604.150984:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574604.854949:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574605.174999:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574605.878957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574606.199038:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788574606.711020:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574606.711028:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788574606.711032:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574606.711040:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6519 00000100:00080000:3.0:1788574606.711050:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574606.711056:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574606.711101:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2300 x1875449616439552/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574606.711125:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574606.711128:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788574606.711129:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574606.711132:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6519 00000100:00080000:3.0:1788574606.711135:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574606.711137:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574606.711146:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574606.711156:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0a80 x1875449616439680/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574606.711164:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1788574606.711166:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616439552:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788574606.711168:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788574606.711172:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574606.711177:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6519 00000100:00080000:3.0:1788574606.711179:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574606.711182:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574606.711185:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574606.711194:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2d80 x1875449616439808/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788574606.711228:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574606.711256:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574606/real 1788574606] req@ffff9ada604e2300 x1875449616439552/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574631 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574606.711288:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2300 x1875449616439552/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574631 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574606.711300:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574606.711304:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574606.711308:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616439552:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574606.711332:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616439680:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574606.711367:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574606.711383:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574606/real 1788574606] req@ffff9ada604e0a80 x1875449616439680/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574631 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574606.711398:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0a80 x1875449616439680/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574631 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574606.711408:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574606.711412:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574606.711415:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616439680:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574606.711422:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616439808:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574606.711452:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574606.711465:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574606/real 1788574606] req@ffff9ada604e2d80 x1875449616439808/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574631 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574606.711475:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2d80 x1875449616439808/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574631 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574606.711481:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574606.711483:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574606.711485:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616439808:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574606.902955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574607.222986:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574607.926918:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574608.247040:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574608.950980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574609.271045:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574609.974931:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574610.294989:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574610.998938:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574611.318973:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788574611.830976:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574611.830992:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788574611.830999:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574611.831013:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6524 00000100:00080000:3.0:1788574611.831027:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574611.831037:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574611.831089:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1880 x1875449616439936/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574611.831122:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574611.831126:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788574611.831130:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574611.831135:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6524 00000100:00080000:3.0:1788574611.831140:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574611.831145:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574611.831157:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574611.831162:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616439936:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574611.831177:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3800 x1875449616440064/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574611.831194:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574611.831199:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788574611.831201:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574611.831206:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6524 00000100:00080000:3.0:1788574611.831210:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574611.831217:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574611.831222:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574611.831236:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1180 x1875449616440192/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788574611.831259:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574611.831306:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574611/real 1788574611] req@ffff9ada604e1880 x1875449616439936/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574636 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574611.831321:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1880 x1875449616439936/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574636 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574611.831330:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574611.831334:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574611.831337:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616439936:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574611.831357:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616440064:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574611.831375:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574611.831383:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574611/real 1788574611] req@ffff9ada604e3800 x1875449616440064/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574636 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574611.831391:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3800 x1875449616440064/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574636 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574611.831405:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574611.831407:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574611.831409:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616440064:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574611.831414:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616440192:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574611.831427:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574611.831439:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574611/real 1788574611] req@ffff9ada604e1180 x1875449616440192/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574636 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574611.831451:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1180 x1875449616440192/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574636 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574611.831459:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574611.831463:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574611.831467:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616440192:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574612.022946:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574612.343046:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574613.047011:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574613.367040:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574614.070902:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574614.391005:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574615.094960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574615.414991:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574616.118963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574616.439005:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788574616.951011:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574616.951021:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788574616.951025:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574616.951034:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6529 00000100:00080000:1.0:1788574616.951048:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574616.951056:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574616.951096:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d180 x1875449616440320/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574616.951121:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574616.951124:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788574616.951125:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574616.951132:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6529 00000100:00080000:1.0:1788574616.951135:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574616.951138:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574616.951147:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574616.951165:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ca80 x1875449616440448/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788574616.951172:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616440320:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788574616.951182:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574616.951187:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788574616.951189:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574616.951194:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6529 00000100:00080000:1.0:1788574616.951199:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574616.951202:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574616.951230:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574616.951242:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d500 x1875449616440576/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788574616.951262:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574616.951301:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574616/real 1788574616] req@ffff9ada4566d180 x1875449616440320/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574641 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574616.951320:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d180 x1875449616440320/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574641 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574616.951328:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574616.951331:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574616.951334:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616440320:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574616.951353:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616440448:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574616.951370:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574616.951379:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574616/real 1788574616] req@ffff9ada4566ca80 x1875449616440448/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574641 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574616.951386:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ca80 x1875449616440448/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574641 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574616.951392:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574616.951394:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574616.951397:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616440448:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574616.951400:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616440576:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574616.951411:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574616.951416:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574616/real 1788574616] req@ffff9ada4566d500 x1875449616440576/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574641 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574616.951424:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d500 x1875449616440576/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574641 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574616.951429:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574616.951431:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574616.951433:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616440576:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574617.142973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574617.463020:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574618.166987:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574618.487056:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574619.190962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574619.511010:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574620.214957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574620.535050:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574621.238956:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574621.559058:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574622.071036:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574622.071047:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574622.071051:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574622.071063:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6534 00000100:00080000:0.0:1788574622.071075:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574622.071083:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574622.071137:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616440704/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574622.071183:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574622.071187:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574622.071188:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574622.071192:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6534 00000100:00080000:0.0:1788574622.071195:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574622.071200:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574622.071211:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574622.071223:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616440832/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574622.071232:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574622.071236:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574622.071239:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574622.071246:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6534 00000100:00100000:3.0:1788574622.071246:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616440704:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788574622.071254:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574622.071265:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574622.071270:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574622.071281:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616440960/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574622.071324:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574622.071356:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574622/real 1788574622] req@ffff9ada4557e300 x1875449616440704/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574647 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574622.071372:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616440704/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574647 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574622.071380:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574622.071383:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574622.071386:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616440704:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574622.071406:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616440832:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574622.071421:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574622.071429:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574622/real 1788574622] req@ffff9ada4557fb80 x1875449616440832/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574647 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574622.071437:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616440832/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574647 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574622.071442:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574622.071445:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574622.071447:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616440832:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574622.071451:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616440960:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574622.071463:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574622.071469:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574622/real 1788574622] req@ffff9ada4557ed80 x1875449616440960/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574647 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574622.071476:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616440960/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574647 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574622.071481:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574622.071483:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574622.071484:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616440960:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574622.262956:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574622.583098:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574623.286975:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574623.607079:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574624.310974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574624.631125:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574625.334978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574625.655079:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574626.358946:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574626.679038:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788574627.191002:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574627.191011:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788574627.191014:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574627.191023:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6539 00000100:00080000:1.0:1788574627.191032:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574627.191038:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574627.191072:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f480 x1875449616441088/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574627.191099:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574627.191103:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788574627.191105:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574627.191109:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6539 00000100:00080000:1.0:1788574627.191113:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574627.191143:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574627.191153:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574627.191166:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e300 x1875449616441216/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574627.191177:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574627.191180:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788574627.191183:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574627.191187:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6539 00000100:00080000:1.0:1788574627.191191:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574627.191196:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574627.191201:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574627.191211:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616441088:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574627.191213:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ea00 x1875449616441344/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574627.191282:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574627.191317:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574627/real 1788574627] req@ffff9ada4566f480 x1875449616441088/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574652 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574627.191338:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f480 x1875449616441088/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574652 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574627.191356:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574627.191360:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574627.191365:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616441088:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574627.191373:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616441216:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574627.191396:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574627.191410:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574627/real 1788574627] req@ffff9ada4566e300 x1875449616441216/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574652 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574627.191424:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e300 x1875449616441216/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574652 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574627.191433:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574627.191436:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574627.191440:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616441216:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574627.191466:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616441344:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574627.191487:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574627.191499:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574627/real 1788574627] req@ffff9ada4566ea00 x1875449616441344/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574652 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574627.191512:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ea00 x1875449616441344/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574652 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574627.191521:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574627.191525:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574627.191528:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616441344:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574627.382955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574627.703037:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574628.406960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574628.726984:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574629.430946:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574629.751057:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574630.454961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574630.775021:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574631.478992:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574631.799033:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788574632.310955:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574632.310965:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788574632.310970:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574632.310980:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6544 00000100:00080000:1.0:1788574632.310991:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574632.310999:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574632.311035:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f800 x1875449616441472/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574632.311076:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574632.311080:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788574632.311082:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574632.311087:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6544 00000100:00080000:1.0:1788574632.311090:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574632.311095:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574632.311106:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574632.311123:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566dc00 x1875449616441600/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574632.311135:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574632.311140:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788574632.311142:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574632.311146:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6544 00000100:00080000:1.0:1788574632.311150:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574632.311158:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574632.311160:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616441472:192.168.202.113@tcp:250:kworker.0 00000008:00100000:1.0:1788574632.311165:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574632.311186:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ed80 x1875449616441728/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574632.311248:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574632.311290:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574632/real 1788574632] req@ffff9ada4566f800 x1875449616441472/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574657 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574632.311308:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f800 x1875449616441472/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574657 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574632.311318:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574632.311321:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574632.311325:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616441472:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574632.311332:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616441600:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574632.311359:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574632.311368:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574632/real 1788574632] req@ffff9ada4566dc00 x1875449616441600/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574657 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574632.311378:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566dc00 x1875449616441600/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574657 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574632.311384:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574632.311386:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574632.311389:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616441600:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574632.311414:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616441728:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574632.311427:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574632.311434:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574632/real 1788574632] req@ffff9ada4566ed80 x1875449616441728/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574657 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574632.311442:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ed80 x1875449616441728/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574657 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574632.311447:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574632.311449:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574632.311451:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616441728:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574632.502985:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574632.823096:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574633.526934:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574633.847037:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574634.551024:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574634.871088:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574635.574934:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574635.895093:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574636.598960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574636.919000:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788574637.430931:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574637.430939:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788574637.430943:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574637.430951:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6549 00000100:00080000:1.0:1788574637.430961:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574637.430967:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574637.430999:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c000 x1875449616441856/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574637.431037:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574637.431041:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788574637.431043:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574637.431048:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6549 00000100:00080000:1.0:1788574637.431052:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574637.431056:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574637.431067:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574637.431077:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616441856:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574637.431081:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566df80 x1875449616441984/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574637.431091:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574637.431095:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788574637.431096:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574637.431100:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6549 00000100:00080000:1.0:1788574637.431103:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574637.431106:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574637.431110:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574637.431118:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566fb80 x1875449616442112/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574637.431135:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574637.431170:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574637/real 1788574637] req@ffff9ada4566c000 x1875449616441856/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574662 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574637.431184:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c000 x1875449616441856/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574662 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574637.431191:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574637.431195:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574637.431197:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616441856:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574637.431216:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616441984:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574637.431233:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574637.431242:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574637/real 1788574637] req@ffff9ada4566df80 x1875449616441984/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574662 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574637.431250:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566df80 x1875449616441984/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574662 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574637.431256:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574637.431258:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574637.431260:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616441984:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574637.431263:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616442112:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574637.431273:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574637.431279:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574637/real 1788574637] req@ffff9ada4566fb80 x1875449616442112/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574662 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574637.431286:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566fb80 x1875449616442112/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574662 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574637.431291:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574637.431293:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574637.431295:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616442112:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574637.622975:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574637.943040:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574638.646983:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574638.967036:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574639.670964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574639.991049:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574640.695016:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574641.015020:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574641.718933:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574642.039027:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788574642.551009:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574642.551020:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788574642.551024:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574642.551035:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6555 00000100:00080000:3.0:1788574642.551046:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574642.551053:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574642.551089:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3480 x1875449616442240/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574642.551121:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574642.551125:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788574642.551126:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574642.551130:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6555 00000100:00080000:3.0:1788574642.551134:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574642.551138:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574642.551148:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574642.551160:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0380 x1875449616442368/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788574642.551167:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616442240:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788574642.551174:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574642.551178:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788574642.551181:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574642.551185:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6555 00000100:00080000:3.0:1788574642.551189:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574642.551193:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574642.551198:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574642.551208:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0700 x1875449616442496/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788574642.551224:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574642.551253:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574642/real 1788574642] req@ffff9ada604e3480 x1875449616442240/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574667 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574642.551267:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3480 x1875449616442240/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574667 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574642.551275:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574642.551278:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574642.551281:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616442240:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574642.551299:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616442368:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574642.551313:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574642.551321:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574642/real 1788574642] req@ffff9ada604e0380 x1875449616442368/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574667 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574642.551329:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0380 x1875449616442368/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574667 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574642.551335:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574642.551337:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574642.551340:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616442368:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574642.551343:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616442496:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574642.551354:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574642.551360:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574642/real 1788574642] req@ffff9ada604e0700 x1875449616442496/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574667 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574642.551367:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0700 x1875449616442496/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574667 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574642.551372:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574642.551373:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574642.551375:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616442496:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574642.742977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574643.063024:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574643.766943:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574644.087012:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574644.790994:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574645.111099:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574645.815016:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574646.135020:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574646.838967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574647.159029:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788574647.670969:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574647.670980:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788574647.670984:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574647.670995:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6560 00000100:00080000:3.0:1788574647.671006:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574647.671014:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574647.671047:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0700 x1875449616442624/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574647.671072:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574647.671075:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788574647.671076:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574647.671079:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6560 00000100:00080000:3.0:1788574647.671082:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574647.671085:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574647.671092:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574647.671101:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0380 x1875449616442752/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574647.671109:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1788574647.671112:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616442624:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788574647.671113:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788574647.671115:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574647.671121:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6560 00000100:00080000:3.0:1788574647.671126:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574647.671128:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574647.671131:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574647.671138:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3480 x1875449616442880/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788574647.671209:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616442752:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574647.671235:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616442880:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574647.862964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574648.182996:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574648.886945:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574649.207033:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574649.911025:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574650.231034:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0:1788574650.743154:0:2371:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:3.0:1788574650.743189:0:2371:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:3.0:1788574650.743203:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:3.0:1788574650.743236:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:3.0:1788574650.743242:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:0.0:1788574650.743323:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574647/real 1788574650] req@ffff9ada604e0380 x1875449616442752/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574672 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574650.743364:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0380 x1875449616442752/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574672 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574650.743379:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574650.743385:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574650.743388:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616442752:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574650.743396:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574647/real 1788574650] req@ffff9ada604e3480 x1875449616442880/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574672 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574650.743425:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3480 x1875449616442880/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574672 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574650.743436:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574650.743441:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574650.743445:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616442880:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574650.743454:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574647/real 1788574650] req@ffff9ada604e0700 x1875449616442624/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574672 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574650.743468:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0700 x1875449616442624/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574672 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574650.743478:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574650.743481:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574650.743485:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616442624:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574650.934977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574651.255075:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574651.958990:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574652.279009:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574652.791009:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574652.791020:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574652.791025:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574652.791035:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6565 00000100:00080000:2.0:1788574652.791045:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574652.791052:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574652.791088:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627800 x1875449616443008/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574652.791114:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574652.791118:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574652.791121:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574652.791125:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6565 00000100:00080000:2.0:1788574652.791129:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574652.791133:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574652.791142:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574652.791156:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626300 x1875449616443136/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574652.791167:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574652.791171:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574652.791173:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574652.791180:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6565 00000100:00100000:0.0:1788574652.791182:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616443008:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788574652.791185:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574652.791194:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574652.791199:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574652.791213:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616443264/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788574652.791254:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574652.791297:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574652/real 1788574652] req@ffff9ada60627800 x1875449616443008/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574677 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574652.791317:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627800 x1875449616443008/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574677 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574652.791328:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574652.791332:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574652.791336:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616443008:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574652.791344:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616443136:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574652.791369:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574652.791383:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574652/real 1788574652] req@ffff9ada60626300 x1875449616443136/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574677 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574652.791394:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626300 x1875449616443136/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574677 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574652.791402:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574652.791405:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574652.791409:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616443136:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574652.791434:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616443264:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574652.791455:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574652.791467:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574652/real 1788574652] req@ffff9ada60627b80 x1875449616443264/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574677 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574652.791479:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616443264/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574677 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574652.791488:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574652.791492:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574652.791495:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616443264:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574652.982931:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574653.303072:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574654.006946:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574654.327016:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574655.030945:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574655.350995:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574656.054965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574656.375146:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574657.078952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574657.399024:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574657.910985:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574657.910996:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574657.911000:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574657.911010:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6570 00000100:00080000:2.0:1788574657.911021:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574657.911030:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574657.911070:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616443392/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574657.911109:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574657.911113:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574657.911115:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574657.911128:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6570 00000100:00080000:2.0:1788574657.911133:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574657.911138:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574657.911148:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574657.911157:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616443392:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788574657.911162:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616443520/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574657.911172:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574657.911174:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574657.911176:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574657.911178:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6570 00000100:00080000:2.0:1788574657.911182:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574657.911184:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574657.911187:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574657.911195:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616443648/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788574657.911218:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574657.911250:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574657/real 1788574657] req@ffff9ada60626d80 x1875449616443392/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574682 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574657.911264:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616443392/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574682 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574657.911273:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574657.911275:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574657.911279:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616443392:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574657.911296:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616443520:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574657.911311:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574657.911319:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574657/real 1788574657] req@ffff9ada60625180 x1875449616443520/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574682 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574657.911327:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616443520/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574682 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574657.911332:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574657.911335:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574657.911337:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616443520:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574657.911340:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616443648:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574657.911350:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574657.911355:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574657/real 1788574657] req@ffff9ada60627480 x1875449616443648/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574682 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574657.911362:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616443648/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574682 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574657.911368:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574657.911369:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574657.911371:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616443648:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574658.102957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574658.422984:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574659.126929:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574659.447044:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574660.151000:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574660.471009:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574661.174933:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574661.495039:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574662.198977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574662.519002:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574663.030948:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574663.030957:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574663.030960:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574663.030969:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6575 00000100:00080000:2.0:1788574663.030978:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574663.030984:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574663.031018:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616443776/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574663.031047:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574663.031050:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574663.031052:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574663.031054:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6575 00000100:00080000:2.0:1788574663.031056:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574663.031059:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574663.031066:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574663.031074:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616443904/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574663.031082:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574663.031084:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574663.031085:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574663.031088:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6575 00000100:00080000:2.0:1788574663.031090:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574663.031092:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574663.031095:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574663.031103:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616444032/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788574663.031106:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616443776:192.168.202.113@tcp:250:kworker.0 00000800:00000100:0.0:1788574663.031178:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574663.031217:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574663/real 1788574663] req@ffff9ada60624e00 x1875449616443776/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574688 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574663.031238:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616443776/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574688 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574663.031250:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574663.031254:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574663.031259:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616443776:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574663.031266:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616443904:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574663.031291:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574663.031316:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574663/real 1788574663] req@ffff9ada60625f80 x1875449616443904/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574688 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574663.031331:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616443904/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574688 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574663.031340:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574663.031343:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574663.031347:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616443904:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574663.031374:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616444032:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574663.031396:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574663.031409:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574663/real 1788574663] req@ffff9ada60625c00 x1875449616444032/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574688 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574663.031421:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616444032/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574688 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574663.031436:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574663.031440:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574663.031443:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616444032:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574663.222976:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574663.543101:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574664.246969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574664.567038:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574665.270930:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574665.590990:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574666.294981:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574666.615047:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574667.318969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574667.639012:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788574668.150983:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574668.150991:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788574668.150995:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574668.151003:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6580 00000100:00080000:1.0:1788574668.151011:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574668.151019:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574668.151061:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566fb80 x1875449616444160/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574668.151085:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574668.151089:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788574668.151091:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574668.151095:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6580 00000100:00080000:1.0:1788574668.151098:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574668.151102:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574668.151112:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574668.151126:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566df80 x1875449616444288/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788574668.151133:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616444160:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788574668.151137:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574668.151140:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788574668.151141:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574668.151145:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6580 00000100:00080000:1.0:1788574668.151148:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574668.151151:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574668.151156:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574668.151165:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c000 x1875449616444416/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788574668.151205:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574668.151238:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574668/real 1788574668] req@ffff9ada4566fb80 x1875449616444160/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574693 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574668.151257:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566fb80 x1875449616444160/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574693 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574668.151269:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574668.151273:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574668.151277:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616444160:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574668.151303:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616444288:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574668.151338:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574668.151351:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574668/real 1788574668] req@ffff9ada4566df80 x1875449616444288/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574693 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574668.151360:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566df80 x1875449616444288/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574693 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574668.151366:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574668.151369:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574668.151371:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616444288:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574668.151376:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616444416:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574668.151398:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574668.151405:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574668/real 1788574668] req@ffff9ada4566c000 x1875449616444416/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574693 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574668.151413:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c000 x1875449616444416/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574693 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574668.151419:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574668.151421:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574668.151423:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616444416:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574668.342935:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574668.663040:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574669.366970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574669.686985:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574670.390933:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574670.711141:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574671.414956:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574671.735024:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574672.438916:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574672.759023:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574673.270965:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574673.270978:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574673.270983:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574673.270993:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6585 00000100:00080000:2.0:1788574673.271004:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574673.271013:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574673.271052:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616444544/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574673.271081:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574673.271085:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574673.271087:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574673.271092:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6585 00000100:00080000:2.0:1788574673.271096:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574673.271100:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574673.271110:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574673.271125:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616444672/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788574673.271136:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616444544:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788574673.271141:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574673.271147:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574673.271149:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574673.271154:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6585 00000100:00080000:2.0:1788574673.271158:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574673.271163:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574673.271168:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574673.271183:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616444800/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788574673.271207:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574673.271242:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574673/real 1788574673] req@ffff9ada60625500 x1875449616444544/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574698 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574673.271254:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616444544/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574698 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574673.271262:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574673.271264:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574673.271268:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616444544:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574673.271291:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616444672:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574673.271316:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574673.271330:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574673/real 1788574673] req@ffff9ada60625880 x1875449616444672/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574698 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574673.271345:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616444672/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574698 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574673.271354:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574673.271358:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574673.271361:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616444672:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574673.271368:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616444800:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574673.271390:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574673.271402:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574673/real 1788574673] req@ffff9ada60624a80 x1875449616444800/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574698 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574673.271416:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616444800/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574698 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574673.271424:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574673.271428:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574673.271431:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616444800:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574673.463012:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574673.783038:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574674.486921:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574674.807022:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574675.510992:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574675.831051:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574676.534966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574676.855013:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574677.558946:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574677.878982:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574678.390907:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574678.390915:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574678.390919:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574678.390926:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6590 00000100:00080000:2.0:1788574678.390934:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574678.390940:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574678.390971:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616444928/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574678.390993:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574678.390995:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574678.390996:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574678.390998:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6590 00000100:00080000:2.0:1788574678.391000:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574678.391002:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574678.391009:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574678.391016:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616445056/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574678.391023:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574678.391025:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574678.391028:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1788574678.391030:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616444928:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788574678.391032:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6590 00000100:00080000:2.0:1788574678.391036:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574678.391039:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574678.391043:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574678.391050:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616445184/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788574678.391081:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574678.391111:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574678/real 1788574678] req@ffff9ada60626a00 x1875449616444928/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574703 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574678.391123:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616444928/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574703 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574678.391134:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574678.391137:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574678.391139:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616444928:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574678.391160:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616445056:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574678.391173:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574678.391181:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574678/real 1788574678] req@ffff9ada60624380 x1875449616445056/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574703 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574678.391188:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616445056/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574703 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574678.391193:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574678.391195:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574678.391197:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616445056:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574678.391200:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616445184:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574678.391209:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574678.391219:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574678/real 1788574678] req@ffff9ada60626680 x1875449616445184/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574703 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574678.391226:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616445184/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574703 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574678.391230:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574678.391232:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574678.391234:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616445184:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574678.582950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574678.902992:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574679.606957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574679.926963:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574680.630937:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574680.951010:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574681.654973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574681.975015:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574682.678945:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574682.999002:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574683.510941:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574683.510949:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574683.510953:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574683.510960:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6596 00000100:00080000:2.0:1788574683.510968:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574683.510974:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574683.511002:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624700 x1875449616445312/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574683.511026:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574683.511028:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574683.511029:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574683.511032:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6596 00000100:00080000:2.0:1788574683.511034:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574683.511035:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574683.511042:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574683.511049:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627100 x1875449616445440/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574683.511055:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574683.511057:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574683.511061:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574683.511063:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6596 00000100:00080000:2.0:1788574683.511066:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574683.511068:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574683.511071:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574683.511081:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616445568/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788574683.511111:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616445312:192.168.202.113@tcp:250:kworker.0 00000800:00000100:0.0:1788574683.511161:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574683.511186:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574683/real 1788574683] req@ffff9ada60624700 x1875449616445312/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574708 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574683.511198:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624700 x1875449616445312/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574708 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574683.511205:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574683.511207:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574683.511210:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616445312:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574683.511214:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616445440:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574683.511225:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574683.511231:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574683/real 1788574683] req@ffff9ada60627100 x1875449616445440/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574708 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574683.511237:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627100 x1875449616445440/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574708 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574683.511242:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574683.511243:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574683.511245:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616445440:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574683.511248:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616445568:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574683.511262:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574683.511270:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574683/real 1788574683] req@ffff9ada60626680 x1875449616445568/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574708 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574683.511277:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616445568/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574708 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574683.511282:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574683.511283:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574683.511285:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616445568:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574683.702926:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574684.022992:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574684.726930:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574685.047234:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574685.750964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574686.071055:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574686.774961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574687.094994:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574687.798972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574688.119052:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788574688.630976:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574688.630985:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788574688.630988:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574688.630996:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6601 00000100:00080000:1.0:1788574688.631005:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574688.631011:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574688.631052:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ed80 x1875449616445696/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574688.631084:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574688.631088:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788574688.631090:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574688.631094:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6601 00000100:00080000:1.0:1788574688.631097:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574688.631100:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574688.631123:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574688.631137:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566dc00 x1875449616445824/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788574688.631145:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616445696:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788574688.631149:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574688.631153:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788574688.631154:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574688.631158:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6601 00000100:00080000:1.0:1788574688.631161:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574688.631164:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574688.631168:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574688.631178:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f800 x1875449616445952/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788574688.631203:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574688.631232:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574688/real 1788574688] req@ffff9ada4566ed80 x1875449616445696/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574713 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574688.631246:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ed80 x1875449616445696/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574713 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574688.631253:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574688.631255:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574688.631258:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616445696:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574688.631280:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616445824:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574688.631304:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574688.631318:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574688/real 1788574688] req@ffff9ada4566dc00 x1875449616445824/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574713 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574688.631330:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566dc00 x1875449616445824/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574713 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574688.631340:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574688.631342:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574688.631345:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616445824:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574688.631349:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616445952:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574688.631370:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574688.631386:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574688/real 1788574688] req@ffff9ada4566f800 x1875449616445952/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574713 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574688.631394:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f800 x1875449616445952/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574713 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574688.631400:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574688.631402:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574688.631404:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616445952:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574688.822961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574689.142996:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574689.846961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574690.167017:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574690.870982:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574691.191060:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574691.894936:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574692.215255:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574692.918989:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574693.239070:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788574693.751003:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574693.751013:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788574693.751018:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574693.751030:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6606 00000100:00080000:1.0:1788574693.751041:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574693.751049:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574693.751090:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ea00 x1875449616446080/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574693.751122:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574693.751127:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788574693.751129:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574693.751133:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6606 00000100:00080000:1.0:1788574693.751137:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574693.751142:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574693.751152:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574693.751168:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e300 x1875449616446208/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788574693.751180:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616446080:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788574693.751183:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574693.751191:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788574693.751193:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574693.751198:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6606 00000100:00080000:1.0:1788574693.751202:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574693.751206:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574693.751211:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574693.751226:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f480 x1875449616446336/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788574693.751255:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574693.751300:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574693/real 1788574693] req@ffff9ada4566ea00 x1875449616446080/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574718 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574693.751315:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ea00 x1875449616446080/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574718 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574693.751324:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574693.751327:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574693.751330:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616446080:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574693.751349:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616446208:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574693.751370:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574693.751383:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574693/real 1788574693] req@ffff9ada4566e300 x1875449616446208/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574718 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574693.751397:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e300 x1875449616446208/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574718 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574693.751407:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574693.751411:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574693.751414:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616446208:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574693.751421:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616446336:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574693.751438:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574693.751446:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574693/real 1788574693] req@ffff9ada4566f480 x1875449616446336/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574718 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574693.751454:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f480 x1875449616446336/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574718 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574693.751459:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574693.751461:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574693.751463:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616446336:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574693.942962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574694.263033:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574694.967006:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574695.287050:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574695.991005:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574696.311036:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574697.014942:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574697.335025:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574698.038973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574698.359009:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788574698.870968:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574698.870976:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788574698.870980:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574698.870988:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6611 00000100:00080000:1.0:1788574698.870998:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574698.871003:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574698.871035:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d500 x1875449616446464/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574698.871059:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574698.871062:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788574698.871063:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574698.871066:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6611 00000100:00080000:1.0:1788574698.871068:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574698.871071:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574698.871078:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574698.871086:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ca80 x1875449616446592/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574698.871093:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574698.871096:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788574698.871097:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574698.871099:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6611 00000100:00080000:1.0:1788574698.871102:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574698.871105:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574698.871107:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574698.871113:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d180 x1875449616446720/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788574698.871130:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616446464:192.168.202.113@tcp:250:kworker.0 00000800:00000100:0.0:1788574698.871227:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574698.871266:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574698/real 1788574698] req@ffff9ada4566d500 x1875449616446464/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574723 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574698.871287:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d500 x1875449616446464/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574723 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574698.871300:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574698.871304:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574698.871308:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616446464:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574698.871317:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616446592:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574698.871343:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574698.871358:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574698/real 1788574698] req@ffff9ada4566ca80 x1875449616446592/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574723 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574698.871373:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ca80 x1875449616446592/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574723 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574698.871382:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574698.871385:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574698.871389:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616446592:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574698.871413:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616446720:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574698.871435:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574698.871447:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574698/real 1788574698] req@ffff9ada4566d180 x1875449616446720/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574723 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574698.871461:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d180 x1875449616446720/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574723 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574698.871477:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574698.871480:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574698.871484:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616446720:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574699.062955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574699.383077:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574700.086942:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574700.407007:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574701.111014:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574701.431022:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574702.134945:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574702.454980:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574703.158966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574703.479069:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788574703.990963:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574703.990974:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788574703.990979:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574703.990990:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6616 00000100:00080000:1.0:1788574703.991000:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574703.991008:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574703.991046:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f100 x1875449616446848/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574703.991067:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574703.991071:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788574703.991073:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574703.991078:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6616 00000100:00080000:1.0:1788574703.991082:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574703.991086:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574703.991096:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574703.991098:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616446848:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574703.991108:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c380 x1875449616446976/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574703.991117:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574703.991120:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788574703.991122:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574703.991125:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6616 00000100:00080000:1.0:1788574703.991128:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574703.991132:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574703.991136:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574703.991144:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c700 x1875449616447104/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788574703.991155:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574703.991188:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574703/real 1788574703] req@ffff9ada4566f100 x1875449616446848/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574728 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574703.991203:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f100 x1875449616446848/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574728 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574703.991218:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574703.991221:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574703.991224:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616446848:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574703.991243:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616446976:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574703.991260:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574703.991268:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574703/real 1788574703] req@ffff9ada4566c380 x1875449616446976/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574728 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574703.991276:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c380 x1875449616446976/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574728 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574703.991282:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574703.991284:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574703.991286:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616446976:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574703.991290:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616447104:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574703.991300:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574703.991306:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574703/real 1788574703] req@ffff9ada4566c700 x1875449616447104/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574728 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574703.991312:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c700 x1875449616447104/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574728 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574703.991317:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574703.991319:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574703.991321:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616447104:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574704.182927:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574704.503036:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574705.206919:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574705.527055:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574706.230938:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574706.551074:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574707.254938:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574707.575068:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574708.279008:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574708.599007:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788574709.110944:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574709.110954:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788574709.110957:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574709.110966:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6621 00000100:00080000:1.0:1788574709.110976:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574709.110982:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574709.111016:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ce00 x1875449616447232/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574709.111055:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574709.111058:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788574709.111060:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574709.111063:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6621 00000100:00080000:1.0:1788574709.111066:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574709.111069:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574709.111077:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574709.111097:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d880 x1875449616447360/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574709.111108:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574709.111112:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788574709.111114:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574709.111118:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6621 00000100:00080000:1.0:1788574709.111121:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574709.111123:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574709.111127:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574709.111138:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e680 x1875449616447488/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788574709.111154:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616447232:192.168.202.113@tcp:250:kworker.0 00000800:00000100:0.0:1788574709.111285:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574709.111345:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574709/real 1788574709] req@ffff9ada4566ce00 x1875449616447232/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574734 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574709.111367:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ce00 x1875449616447232/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574734 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574709.111381:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574709.111385:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574709.111390:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616447232:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574709.111402:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616447360:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574709.111425:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574709.111442:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574709/real 1788574709] req@ffff9ada4566d880 x1875449616447360/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574734 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574709.111455:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d880 x1875449616447360/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574734 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574709.111464:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574709.111468:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574709.111471:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616447360:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574709.111499:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616447488:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574709.111515:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574709.111524:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574709/real 1788574709] req@ffff9ada4566e680 x1875449616447488/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574734 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574709.111536:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e680 x1875449616447488/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574734 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574709.111545:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574709.111548:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574709.111551:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616447488:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574709.302949:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574709.623040:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574710.326969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574710.646991:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574711.350907:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574711.671003:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574712.374982:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574712.695051:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574713.398968:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574713.719038:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574714.230992:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574714.231000:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574714.231004:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574714.231014:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6626 00000100:00080000:2.0:1788574714.231023:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574714.231028:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574714.231063:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616447616/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574714.231085:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574714.231088:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574714.231090:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574714.231092:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6626 00000100:00080000:2.0:1788574714.231095:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574714.231097:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574714.231105:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574714.231113:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616447744/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574714.231122:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574714.231126:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574714.231129:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1788574714.231130:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616447616:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788574714.231134:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6626 00000100:00080000:2.0:1788574714.231141:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574714.231147:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574714.231152:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574714.231164:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616447872/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788574714.231218:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616447744:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574714.231245:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616447872:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574714.423002:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574714.742980:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574715.446931:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574715.767047:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574716.470913:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574716.791035:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1788574717.303078:0:2370:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:2.0:1788574717.303124:0:2370:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1788574717.303141:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788574717.303182:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788574717.303193:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:0.0:1788574717.303249:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574714/real 1788574717] req@ffff9ada60626a00 x1875449616447744/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574739 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574717.303276:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616447744/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574739 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574717.303288:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574717.303293:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574717.303297:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616447744:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574717.303305:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574714/real 1788574717] req@ffff9ada60624a80 x1875449616447872/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574739 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574717.303316:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616447872/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574739 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574717.303324:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574717.303328:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574717.303332:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616447872:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574717.303338:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574714/real 1788574717] req@ffff9ada60624380 x1875449616447616/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574739 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574717.303358:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616447616/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574739 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574717.303366:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574717.303369:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574717.303372:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616447616:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574717.494932:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574717.815012:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574718.519007:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574718.839039:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574719.351010:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574719.351021:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574719.351025:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574719.351035:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6631 00000100:00080000:0.0:1788574719.351047:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574719.351055:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574719.351094:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616448000/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574719.351131:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574719.351135:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574719.351137:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574719.351142:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6631 00000100:00080000:0.0:1788574719.351151:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574719.351156:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574719.351166:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574719.351181:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616448128/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574719.351191:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616448000:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788574719.351209:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574719.351213:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574719.351215:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574719.351221:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6631 00000100:00080000:0.0:1788574719.351225:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574719.351229:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574719.351235:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574719.351248:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557df80 x1875449616448256/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574719.351255:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574719.351367:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574719/real 1788574719] req@ffff9ada4557ea00 x1875449616448000/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574744 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574719.351389:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616448000/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574744 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574719.351400:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574719.351404:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574719.351408:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616448000:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574719.351433:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616448128:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574719.351458:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574719.351472:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574719/real 1788574719] req@ffff9ada4557e680 x1875449616448128/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574744 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574719.351485:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616448128/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574744 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574719.351495:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574719.351498:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574719.351501:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616448128:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574719.351508:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616448256:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574719.351528:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574719.351539:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574719/real 1788574719] req@ffff9ada4557df80 x1875449616448256/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574744 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574719.351551:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557df80 x1875449616448256/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574744 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574719.351559:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574719.351562:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574719.351578:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616448256:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574719.542926:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574719.863025:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574720.566967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574720.887037:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574721.590936:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574721.910987:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574722.614965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574722.935033:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574723.638955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574723.959030:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574724.470960:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574724.470969:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574724.470973:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574724.470983:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6637 00000100:00080000:2.0:1788574724.470992:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574724.471005:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574724.471044:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616448384/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574724.471073:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574724.471084:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574724.471087:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574724.471093:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6637 00000100:00080000:2.0:1788574724.471098:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574724.471103:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574724.471110:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616448384:192.168.202.113@tcp:250:kworker.0 00000008:00100000:2.0:1788574724.471112:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574724.471127:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616448512/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574724.471137:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574724.471139:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574724.471141:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574724.471144:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6637 00000100:00080000:2.0:1788574724.471147:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574724.471149:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574724.471152:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574724.471162:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616448640/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574724.471182:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574724.471218:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574724/real 1788574724] req@ffff9ada60625880 x1875449616448384/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574749 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574724.471238:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616448384/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574749 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574724.471249:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574724.471254:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574724.471258:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616448384:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574724.471283:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616448512:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574724.471311:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574724.471334:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574724/real 1788574724] req@ffff9ada60625500 x1875449616448512/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574749 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574724.471350:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616448512/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574749 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574724.471361:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574724.471365:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574724.471368:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616448512:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574724.471375:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616448640:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574724.471397:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574724.471409:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574724/real 1788574724] req@ffff9ada60625c00 x1875449616448640/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574749 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574724.471419:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616448640/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574749 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574724.471427:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574724.471429:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574724.471432:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616448640:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574724.662940:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574724.983003:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574725.686960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574726.007024:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574726.710950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574727.030957:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574727.734984:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574728.054990:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574728.758940:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574729.079063:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574729.590906:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574729.590916:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574729.590921:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574729.590932:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6642 00000100:00080000:2.0:1788574729.590943:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574729.590951:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574729.590985:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616448768/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574729.591010:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574729.591018:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574729.591020:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574729.591022:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6642 00000100:00080000:2.0:1788574729.591025:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574729.591028:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574729.591036:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574729.591046:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616448896/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574729.591055:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574729.591058:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1788574729.591058:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616448768:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788574729.591061:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574729.591070:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6642 00000100:00080000:2.0:1788574729.591072:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574729.591074:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574729.591077:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574729.591086:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616449024/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574729.591129:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574729.591161:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574729/real 1788574729] req@ffff9ada60625f80 x1875449616448768/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574754 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574729.591176:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616448768/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574754 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574729.591184:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574729.591187:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574729.591190:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616448768:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574729.591208:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616448896:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574729.591224:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574729.591232:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574729/real 1788574729] req@ffff9ada60624e00 x1875449616448896/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574754 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574729.591240:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616448896/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574754 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574729.591246:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574729.591248:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574729.591251:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616448896:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574729.591255:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616449024:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574729.591265:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574729.591270:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574729/real 1788574729] req@ffff9ada60627480 x1875449616449024/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574754 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574729.591283:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616449024/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574754 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574729.591288:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574729.591290:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574729.591292:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616449024:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574729.782951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574730.103039:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574730.806923:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574731.127070:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574731.830957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574732.151015:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574732.854912:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574733.175006:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574733.878934:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574734.199061:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574734.711011:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574734.711022:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574734.711026:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574734.711039:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6647 00000100:00080000:2.0:1788574734.711050:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574734.711058:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574734.711099:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616449152/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574734.711133:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574734.711137:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574734.711139:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574734.711143:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6647 00000100:00080000:2.0:1788574734.711147:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574734.711151:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574734.711160:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574734.711172:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616449280/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574734.711184:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574734.711188:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574734.711190:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574734.711194:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6647 00000100:00080000:2.0:1788574734.711198:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574734.711202:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574734.711207:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574734.711211:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616449152:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788574734.711221:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616449408/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574734.711279:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574734.711310:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574734/real 1788574734] req@ffff9ada60625180 x1875449616449152/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574759 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574734.711325:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616449152/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574759 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574734.711335:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574734.711339:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574734.711344:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616449152:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574734.711351:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616449280:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574734.711384:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574734.711398:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574734/real 1788574734] req@ffff9ada60626d80 x1875449616449280/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574759 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574734.711412:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616449280/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574759 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574734.711419:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574734.711422:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574734.711424:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616449280:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574734.711445:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616449408:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574734.711460:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574734.711470:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574734/real 1788574734] req@ffff9ada60627b80 x1875449616449408/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574759 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574734.711482:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616449408/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574759 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574734.711490:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574734.711494:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574734.711497:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616449408:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574734.902946:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574735.223140:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574735.926956:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574736.247051:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574736.950978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574737.271032:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574737.974967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574738.295004:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574738.999053:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574739.319080:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788574739.831012:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574739.831027:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788574739.831033:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574739.831049:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6652 00000100:00080000:3.0:1788574739.831064:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574739.831075:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574739.831129:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1180 x1875449616449536/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574739.831165:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574739.831171:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788574739.831173:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574739.831177:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6652 00000100:00080000:3.0:1788574739.831181:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574739.831185:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574739.831199:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574739.831213:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3800 x1875449616449664/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574739.831225:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574739.831230:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788574739.831232:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574739.831237:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6652 00000100:00080000:3.0:1788574739.831242:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574739.831252:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574739.831259:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616449536:192.168.202.113@tcp:250:kworker.0 00000008:00100000:3.0:1788574739.831263:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574739.831287:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1880 x1875449616449792/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788574739.831345:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574739.831387:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574739/real 1788574739] req@ffff9ada604e1180 x1875449616449536/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574764 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574739.831402:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1180 x1875449616449536/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574764 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574739.831415:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574739.831419:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574739.831422:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616449536:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574739.831432:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616449664:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574739.831448:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574739.831457:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574739/real 1788574739] req@ffff9ada604e3800 x1875449616449664/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574764 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574739.831465:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3800 x1875449616449664/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574764 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574739.831471:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574739.831473:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574739.831487:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616449664:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574739.831508:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616449792:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574739.831522:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574739.831529:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574739/real 1788574739] req@ffff9ada604e1880 x1875449616449792/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574764 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574739.831537:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1880 x1875449616449792/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574764 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574739.831542:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574739.831551:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574739.831553:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616449792:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574740.023005:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574740.342965:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574741.046961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574741.367090:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574742.070969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574742.390987:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574743.094955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574743.415050:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574744.118985:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574744.439012:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574744.951003:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574744.951024:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574744.951033:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574744.951045:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6657 00000100:00080000:2.0:1788574744.951056:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574744.951063:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574744.951097:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626300 x1875449616449920/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574744.951121:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574744.951124:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574744.951125:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574744.951128:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6657 00000100:00080000:2.0:1788574744.951130:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574744.951132:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574744.951140:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574744.951147:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627800 x1875449616450048/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574744.951155:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574744.951157:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574744.951165:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574744.951168:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6657 00000100:00080000:2.0:1788574744.951171:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574744.951175:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574744.951181:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574744.951189:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616450176/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788574744.951220:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616449920:192.168.202.113@tcp:250:kworker.0 00000800:00000100:0.0:1788574744.951303:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574744.951345:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574744/real 1788574744] req@ffff9ada60626300 x1875449616449920/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574769 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574744.951363:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626300 x1875449616449920/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574769 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574744.951372:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574744.951376:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574744.951380:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616449920:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574744.951386:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616450048:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574744.951402:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574744.951410:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574744/real 1788574744] req@ffff9ada60627800 x1875449616450048/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574769 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574744.951422:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627800 x1875449616450048/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574769 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574744.951431:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574744.951435:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574744.951439:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616450048:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574744.951447:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616450176:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574744.951473:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574744.951483:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574744/real 1788574744] req@ffff9ada60627b80 x1875449616450176/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574769 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574744.951497:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616450176/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574769 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574744.951506:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574744.951510:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574744.951514:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616450176:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574745.142934:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574745.463041:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574746.166957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574746.487069:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574747.190961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574747.510983:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574748.214962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574748.535073:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574749.238955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574749.559223:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574750.070965:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574750.070974:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574750.070977:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574750.070986:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6662 00000100:00080000:2.0:1788574750.070995:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574750.071002:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574750.071043:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616450304/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574750.071072:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574750.071075:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574750.071076:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574750.071079:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6662 00000100:00080000:2.0:1788574750.071083:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574750.071087:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574750.071097:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574750.071113:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616450432/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788574750.071117:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616450304:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788574750.071134:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574750.071136:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574750.071138:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574750.071142:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6662 00000100:00080000:2.0:1788574750.071145:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574750.071148:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574750.071153:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574750.071163:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616450560/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788574750.071181:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574750.071212:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574750/real 1788574750] req@ffff9ada60626d80 x1875449616450304/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574775 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574750.071226:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616450304/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574775 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574750.071234:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574750.071237:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574750.071240:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616450304:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574750.071258:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616450432:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574750.071273:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574750.071281:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574750/real 1788574750] req@ffff9ada60625180 x1875449616450432/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574775 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574750.071290:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616450432/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574775 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574750.071305:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574750.071310:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574750.071312:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616450432:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574750.071317:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616450560:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574750.071329:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574750.071335:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574750/real 1788574750] req@ffff9ada60627480 x1875449616450560/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574775 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574750.071343:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616450560/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574775 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574750.071348:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574750.071350:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574750.071352:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616450560:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574750.262917:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574750.583054:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574751.286996:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574751.607026:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574752.310959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574752.631061:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574753.335011:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574753.655019:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574754.358973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574754.679063:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574755.190966:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574755.190977:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574755.190981:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574755.190991:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6667 00000100:00080000:2.0:1788574755.191002:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574755.191009:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574755.191041:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616450688/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574755.191066:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574755.191069:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574755.191070:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574755.191073:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6667 00000100:00080000:2.0:1788574755.191075:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574755.191078:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574755.191085:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574755.191093:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616450816/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574755.191099:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574755.191101:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574755.191103:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574755.191105:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6667 00000100:00080000:2.0:1788574755.191107:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574755.191110:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574755.191113:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574755.191120:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616450944/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788574755.191139:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616450688:192.168.202.113@tcp:250:kworker.0 00000800:00000100:0.0:1788574755.191204:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574755.191234:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574755/real 1788574755] req@ffff9ada60624e00 x1875449616450688/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574780 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574755.191253:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616450688/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574780 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574755.191260:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574755.191263:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574755.191266:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616450688:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574755.191271:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616450816:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574755.191285:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574755.191293:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574755/real 1788574755] req@ffff9ada60625f80 x1875449616450816/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574780 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574755.191301:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616450816/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574780 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574755.191306:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574755.191308:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574755.191310:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616450816:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574755.191328:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616450944:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574755.191339:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574755.191345:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574755/real 1788574755] req@ffff9ada60625c00 x1875449616450944/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574780 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574755.191353:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616450944/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574780 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574755.191362:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574755.191364:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574755.191366:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616450944:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574755.382967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574755.703077:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574756.406988:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574756.727367:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574757.431011:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574757.751039:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574758.454918:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574758.775030:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574759.478983:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574759.799051:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788574760.311029:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574760.311042:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788574760.311047:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574760.311060:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6672 00000100:00080000:3.0:1788574760.311076:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574760.311087:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574760.311146:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2d80 x1875449616451072/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574760.311184:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574760.311188:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788574760.311191:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574760.311196:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6672 00000100:00080000:3.0:1788574760.311201:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574760.311206:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574760.311220:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574760.311239:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0a80 x1875449616451200/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574760.311253:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574760.311257:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788574760.311259:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574760.311265:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6672 00000100:00080000:3.0:1788574760.311268:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574760.311277:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574760.311289:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616451072:192.168.202.113@tcp:250:kworker.0 00000008:00100000:3.0:1788574760.311289:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574760.311325:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2300 x1875449616451328/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788574760.311395:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574760.311437:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574760/real 1788574760] req@ffff9ada604e2d80 x1875449616451072/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574785 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574760.311456:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2d80 x1875449616451072/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574785 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574760.311469:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574760.311474:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574760.311478:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616451072:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574760.311487:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616451200:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574760.311513:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574760.311527:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574760/real 1788574760] req@ffff9ada604e0a80 x1875449616451200/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574785 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574760.311540:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0a80 x1875449616451200/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574785 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574760.311550:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574760.311554:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574760.311558:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616451200:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574760.311581:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616451328:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574760.311597:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574760.311606:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574760/real 1788574760] req@ffff9ada604e2300 x1875449616451328/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574785 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574760.311614:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2300 x1875449616451328/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574785 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574760.311619:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574760.311629:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574760.311632:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616451328:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574760.502957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574760.823016:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574761.526971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574761.847024:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574762.550923:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574762.871067:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574763.574976:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574763.895114:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574764.598963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574764.918997:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574765.430994:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574765.431003:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574765.431008:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574765.431018:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6677 00000100:00080000:0.0:1788574765.431027:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574765.431034:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574765.431072:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d500 x1875449616451456/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574765.431105:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574765.431109:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574765.431111:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574765.431115:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6677 00000100:00080000:0.0:1788574765.431120:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574765.431125:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574765.431135:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574765.431149:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d180 x1875449616451584/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574765.431175:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574765.431179:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574765.431184:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1788574765.431184:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616451456:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788574765.431194:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6677 00000100:00080000:0.0:1788574765.431199:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574765.431204:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574765.431210:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574765.431226:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557df80 x1875449616451712/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574765.431254:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574765.431286:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574765/real 1788574765] req@ffff9ada4557d500 x1875449616451456/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574790 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574765.431307:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d500 x1875449616451456/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574790 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574765.431319:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574765.431323:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574765.431327:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616451456:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574765.431352:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616451584:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574765.431375:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574765.431386:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574765/real 1788574765] req@ffff9ada4557d180 x1875449616451584/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574790 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574765.431395:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d180 x1875449616451584/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574790 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574765.431401:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574765.431403:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574765.431405:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616451584:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574765.431411:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616451712:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574765.431424:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574765.431430:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574765/real 1788574765] req@ffff9ada4557df80 x1875449616451712/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574790 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574765.431437:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557df80 x1875449616451712/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574790 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574765.431443:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574765.431444:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574765.431446:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616451712:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574765.622971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574765.943051:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574766.646975:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574766.967004:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574767.670963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574767.991054:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574768.694950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574769.015023:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574769.718967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574770.039059:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788574770.551129:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574770.551220:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788574770.551254:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574770.551303:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6683 00000100:00080000:1.0:1788574770.551354:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574770.551381:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574770.551457:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e680 x1875449616451840/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574770.551540:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1788574770.551557:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616451840:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788574770.551580:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788574770.551603:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574770.551611:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6683 00000100:00080000:1.0:1788574770.551631:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1788574770.551648:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00080000:1.0:1788574770.551679:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574770.551696:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574770/real 1788574770] req@ffff9ada4566e680 x1875449616451840/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574795 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:1.0:1788574770.551701:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574770.551718:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e680 x1875449616451840/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574795 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788574770.551728:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d880 x1875449616451968/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574770.551745:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574770.551754:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574770.551760:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616451840:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788574770.551777:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574770.551781:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1788574770.551804:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616451968:192.168.202.113@tcp:38:kworker.0 00000100:00080000:1.0:1788574770.551825:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574770.551832:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6683 00000800:00000100:3.0:1788574770.551835:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00080000:1.0:1788574770.551846:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1788574770.551849:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574770/real 1788574770] req@ffff9ada4566d880 x1875449616451968/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574795 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1788574770.551893:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574770.551899:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d880 x1875449616451968/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574795 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000008:00100000:1.0:1788574770.551900:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00080000:3.0:1788574770.551916:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:1.0:1788574770.551920:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ce00 x1875449616452096/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574770.551925:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574770.551934:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616451968:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574770.551955:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616452096:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574770.551975:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574770.551985:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574770/real 1788574770] req@ffff9ada4566ce00 x1875449616452096/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574795 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574770.551993:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ce00 x1875449616452096/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574795 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574770.552015:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574770.552018:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574770.552021:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616452096:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574770.743008:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574771.063125:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574771.766990:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574772.087127:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574772.790965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574773.111055:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574773.814986:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574774.135067:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574774.838965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574775.159043:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788574775.671069:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574775.671080:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788574775.671084:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574775.671095:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6688 00000100:00080000:3.0:1788574775.671104:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574775.671111:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574775.671149:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0000 x1875449616452224/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574775.671184:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574775.671188:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788574775.671190:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574775.671193:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6688 00000100:00080000:3.0:1788574775.671196:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574775.671200:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574775.671210:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574775.671222:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1c00 x1875449616452352/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574775.671231:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574775.671233:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788574775.671235:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574775.671238:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6688 00000100:00080000:3.0:1788574775.671241:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574775.671244:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574775.671249:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574775.671258:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3b80 x1875449616452480/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788574775.671348:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616452224:192.168.202.113@tcp:250:kworker.0 00000800:00000100:0.0:1788574775.671404:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574775.671434:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574775/real 1788574775] req@ffff9ada604e0000 x1875449616452224/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574800 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574775.671450:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0000 x1875449616452224/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574800 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574775.671460:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574775.671464:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574775.671468:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616452224:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574775.671475:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616452352:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574775.671494:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574775.671506:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574775/real 1788574775] req@ffff9ada604e1c00 x1875449616452352/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574800 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574775.671518:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1c00 x1875449616452352/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574800 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574775.671525:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574775.671528:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574775.671531:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616452352:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574775.671536:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616452480:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574775.671554:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574775.671564:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574775/real 1788574775] req@ffff9ada604e3b80 x1875449616452480/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574800 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574775.671575:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3b80 x1875449616452480/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574800 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574775.671583:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574775.671586:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574775.671589:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616452480:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574775.862950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574776.183005:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574776.886981:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574777.207013:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574777.910993:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574778.231287:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574778.934956:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574779.254989:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574779.958976:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574780.279023:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788574780.790984:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574780.791004:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788574780.791009:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574780.791020:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6693 00000100:00080000:3.0:1788574780.791034:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574780.791044:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574780.791087:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3100 x1875449616452608/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574780.791119:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574780.791123:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788574780.791125:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574780.791129:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6693 00000100:00080000:3.0:1788574780.791133:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574780.791137:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574780.791149:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574780.791162:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1f80 x1875449616452736/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574780.791173:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1788574780.791174:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616452608:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788574780.791180:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788574780.791184:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574780.791189:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6693 00000100:00080000:3.0:1788574780.791193:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574780.791197:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574780.791201:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574780.791213:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2680 x1875449616452864/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788574780.791288:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616452736:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574780.791318:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616452864:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574780.982961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574781.303078:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574782.006946:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574782.327024:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574783.030956:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574783.351000:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1788574783.863148:0:2372:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:1.0:1788574783.863190:0:2372:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1788574783.863222:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:1.0:1788574783.863256:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:1.0:1788574783.863265:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:0.0:1788574783.863352:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574780/real 1788574783] req@ffff9ada604e1f80 x1875449616452736/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574805 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574783.863380:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1f80 x1875449616452736/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574805 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574783.863392:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574783.863396:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574783.863400:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616452736:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574783.863407:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574780/real 1788574783] req@ffff9ada604e2680 x1875449616452864/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574805 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574783.863417:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2680 x1875449616452864/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574805 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574783.863424:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574783.863427:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574783.863430:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616452864:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574783.863436:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574780/real 1788574783] req@ffff9ada604e3100 x1875449616452608/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574805 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574783.863447:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3100 x1875449616452608/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574805 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574783.863455:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574783.863458:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574783.863461:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616452608:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574784.054953:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574784.375004:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574785.079030:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574785.399099:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574785.911011:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574785.911037:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574785.911042:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574785.911056:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6698 00000100:00080000:2.0:1788574785.911069:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574785.911078:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574785.911127:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616452992/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574785.911169:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574785.911173:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574785.911174:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574785.911178:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6698 00000100:00080000:2.0:1788574785.911182:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574785.911186:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574785.911199:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574785.911217:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616453120/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574785.911226:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574785.911229:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574785.911231:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574785.911235:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6698 00000100:00080000:2.0:1788574785.911240:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574785.911244:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574785.911248:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574785.911258:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616453248/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788574785.911564:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616452992:192.168.202.113@tcp:250:kworker.0 00000800:00000100:0.0:1788574785.911684:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574785.911735:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574785/real 1788574785] req@ffff9ada60625500 x1875449616452992/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574810 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574785.911759:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616452992/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574810 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574785.911771:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574785.911776:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574785.911781:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616452992:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574785.911788:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616453120:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574785.911812:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574785.911824:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574785/real 1788574785] req@ffff9ada60625880 x1875449616453120/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574810 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574785.911835:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616453120/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574810 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574785.911842:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574785.911845:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574785.911847:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616453120:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574785.911895:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616453248:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574785.911920:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574785.911931:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574785/real 1788574785] req@ffff9ada60624380 x1875449616453248/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574810 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574785.911943:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616453248/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574810 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574785.911949:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574785.911952:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574785.911955:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616453248:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574786.102991:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574786.423035:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574787.126949:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574787.447019:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574788.150971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574788.471071:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574789.174967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574789.495065:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574790.198960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574790.519030:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788574791.031038:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574791.031051:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788574791.031056:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574791.031069:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6703 00000100:00080000:3.0:1788574791.031081:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574791.031090:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574791.031143:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2a00 x1875449616453376/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574791.031173:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574791.031176:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788574791.031178:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574791.031182:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6703 00000100:00080000:3.0:1788574791.031186:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574791.031189:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574791.031203:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574791.031220:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1500 x1875449616453504/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574791.031231:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574791.031234:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788574791.031237:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574791.031247:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6703 00000100:00080000:3.0:1788574791.031251:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1788574791.031259:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616453376:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788574791.031260:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574791.031276:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574791.031295:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0e00 x1875449616453632/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788574791.031352:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574791.031397:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574791/real 1788574791] req@ffff9ada604e2a00 x1875449616453376/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574816 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574791.031411:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2a00 x1875449616453376/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574816 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574791.031420:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574791.031423:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574791.031425:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616453376:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574791.031431:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616453504:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574791.031447:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574791.031454:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574791/real 1788574791] req@ffff9ada604e1500 x1875449616453504/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574816 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574791.031466:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1500 x1875449616453504/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574816 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574791.031470:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574791.031472:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574791.031474:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616453504:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574791.031493:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616453632:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574791.031503:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574791.031509:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574791/real 1788574791] req@ffff9ada604e0e00 x1875449616453632/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574816 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574791.031515:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0e00 x1875449616453632/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574816 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574791.031520:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574791.031521:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574791.031523:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616453632:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574791.222916:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574791.543134:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574792.246959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574792.567138:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574793.270938:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574793.591050:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574794.294965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574794.615000:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574795.318957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574795.639031:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788574796.151012:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574796.151023:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788574796.151028:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574796.151040:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6708 00000100:00080000:3.0:1788574796.151053:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574796.151061:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574796.151117:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0e00 x1875449616453760/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574796.151142:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574796.151145:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788574796.151146:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574796.151149:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6708 00000100:00080000:3.0:1788574796.151152:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574796.151154:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574796.151165:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574796.151174:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1500 x1875449616453888/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574796.151181:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574796.151185:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788574796.151186:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574796.151189:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6708 00000100:00080000:3.0:1788574796.151192:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574796.151195:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574796.151198:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574796.151206:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2a00 x1875449616454016/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788574796.151258:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616453760:192.168.202.113@tcp:250:kworker.0 00000800:00000100:0.0:1788574796.151338:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574796.151379:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574796/real 1788574796] req@ffff9ada604e0e00 x1875449616453760/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574821 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574796.151407:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0e00 x1875449616453760/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574821 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574796.151420:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574796.151423:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574796.151427:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616453760:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574796.151433:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616453888:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574796.151454:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574796.151463:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574796/real 1788574796] req@ffff9ada604e1500 x1875449616453888/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574821 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574796.151472:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1500 x1875449616453888/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574821 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574796.151477:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574796.151480:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574796.151482:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616453888:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574796.151502:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616454016:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574796.151514:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574796.151520:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574796/real 1788574796] req@ffff9ada604e2a00 x1875449616454016/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574821 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574796.151528:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2a00 x1875449616454016/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574821 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574796.151537:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574796.151539:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574796.151541:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616454016:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574796.343032:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574796.663070:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574797.366964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574797.687094:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574798.390978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574798.710977:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574799.415003:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574799.735182:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574800.439031:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574800.759109:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574801.271001:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574801.271013:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574801.271018:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574801.271029:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6713 00000100:00080000:2.0:1788574801.271043:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574801.271051:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574801.271129:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616454144/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574801.271165:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574801.271169:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574801.271170:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574801.271174:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6713 00000100:00080000:2.0:1788574801.271177:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574801.271198:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574801.271208:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574801.271222:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616454144:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788574801.271223:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616454272/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574801.271239:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574801.271242:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574801.271244:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574801.271247:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6713 00000100:00080000:2.0:1788574801.271250:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574801.271252:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574801.271256:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574801.271268:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616454400/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788574801.271311:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574801.271367:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574801/real 1788574801] req@ffff9ada60624a80 x1875449616454144/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574826 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574801.271391:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616454144/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574826 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574801.271404:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574801.271409:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574801.271414:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616454144:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574801.271453:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616454272:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574801.271481:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574801.271496:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574801/real 1788574801] req@ffff9ada60626a00 x1875449616454272/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574826 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574801.271506:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616454272/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574826 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574801.271522:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574801.271526:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574801.271530:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616454272:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574801.271534:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616454400:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574801.271563:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574801.271571:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574801/real 1788574801] req@ffff9ada60626680 x1875449616454400/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574826 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574801.271580:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616454400/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574826 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574801.271585:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574801.271588:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574801.271590:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616454400:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574801.462939:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574801.783108:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574802.486988:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574802.807281:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574803.510981:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574803.831075:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574804.534993:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574804.855115:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574805.559005:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574805.879007:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788574806.391000:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574806.391009:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788574806.391013:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574806.391023:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6718 00000100:00080000:3.0:1788574806.391033:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574806.391039:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574806.391085:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3100 x1875449616454528/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574806.391118:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574806.391123:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788574806.391125:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574806.391129:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6718 00000100:00080000:3.0:1788574806.391133:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574806.391139:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574806.391150:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574806.391165:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2680 x1875449616454656/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574806.391180:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574806.391182:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788574806.391184:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574806.391189:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6718 00000100:00100000:0.0:1788574806.391193:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616454528:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788574806.391193:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574806.391207:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574806.391213:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574806.391227:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1f80 x1875449616454784/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788574806.391274:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574806.391313:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574806/real 1788574806] req@ffff9ada604e3100 x1875449616454528/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574831 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574806.391354:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3100 x1875449616454528/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574831 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574806.391374:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574806.391380:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574806.391385:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616454528:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574806.391411:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616454656:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574806.391438:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574806.391453:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574806/real 1788574806] req@ffff9ada604e2680 x1875449616454656/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574831 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574806.391468:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2680 x1875449616454656/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574831 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574806.391477:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574806.391481:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574806.391485:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616454656:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574806.391491:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616454784:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574806.391513:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574806.391525:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574806/real 1788574806] req@ffff9ada604e1f80 x1875449616454784/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574831 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574806.391539:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1f80 x1875449616454784/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574831 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574806.391547:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574806.391551:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574806.391555:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616454784:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574806.582985:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574806.903287:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574807.606981:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574807.927080:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574808.630951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574808.951067:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574809.654981:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574809.975010:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574810.678948:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574810.999013:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574811.511013:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574811.511023:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574811.511027:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574811.511037:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6724 00000100:00080000:0.0:1788574811.511047:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574811.511054:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574811.511088:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616454912/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574811.511121:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574811.511125:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574811.511126:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574811.511130:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6724 00000100:00080000:0.0:1788574811.511134:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574811.511138:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574811.511147:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574811.511159:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616455040/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574811.511171:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574811.511178:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574811.511182:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1788574811.511182:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616454912:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788574811.511193:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6724 00000100:00080000:0.0:1788574811.511199:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574811.511203:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574811.511207:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574811.511228:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616455168/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574811.511288:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574811.511344:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574811/real 1788574811] req@ffff9ada4557e680 x1875449616454912/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574836 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574811.511364:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616454912/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574836 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574811.511373:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574811.511376:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574811.511380:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616454912:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574811.511417:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616455040:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574811.511445:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574811.511458:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574811/real 1788574811] req@ffff9ada4557ea00 x1875449616455040/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574836 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574811.511470:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616455040/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574836 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574811.511479:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574811.511483:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574811.511486:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616455040:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574811.511491:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616455168:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574811.511510:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574811.511521:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574811/real 1788574811] req@ffff9ada4557ed80 x1875449616455168/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574836 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574811.511533:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616455168/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574836 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574811.511541:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574811.511551:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574811.511554:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616455168:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574811.703011:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574812.023017:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574812.726983:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574813.047018:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574813.750976:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574814.071165:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574814.775011:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574815.095034:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574815.798953:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574816.119011:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574816.630921:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574816.630928:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574816.630932:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574816.630939:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6729 00000100:00080000:0.0:1788574816.630948:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574816.630953:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574816.630985:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616455296/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574816.631008:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574816.631011:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574816.631012:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574816.631018:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6729 00000100:00080000:0.0:1788574816.631020:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574816.631023:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574816.631030:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574816.631038:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616455424/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574816.631045:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574816.631047:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574816.631048:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574816.631051:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6729 00000100:00080000:0.0:1788574816.631052:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1788574816.631056:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616455296:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788574816.631057:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574816.631064:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574816.631070:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616455552/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574816.631113:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574816.631139:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574816/real 1788574816] req@ffff9ada4557fb80 x1875449616455296/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574841 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574816.631152:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616455296/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574841 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574816.631159:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574816.631162:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574816.631164:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616455296:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574816.631169:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616455424:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574816.631181:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574816.631187:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574816/real 1788574816] req@ffff9ada4557e300 x1875449616455424/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574841 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574816.631194:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616455424/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574841 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574816.631199:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574816.631201:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574816.631202:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616455424:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574816.631219:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616455552:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574816.631229:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574816.631234:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574816/real 1788574816] req@ffff9ada4557dc00 x1875449616455552/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574841 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574816.631241:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616455552/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574841 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574816.631245:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574816.631247:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574816.631249:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616455552:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574816.822971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574817.143009:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574817.846968:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574818.166997:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574818.870965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574819.191006:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574819.894958:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574820.214996:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574820.918938:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574821.239043:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788574821.750980:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574821.750988:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788574821.750992:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574821.751000:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6734 00000100:00080000:1.0:1788574821.751008:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574821.751014:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574821.751046:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c700 x1875449616455680/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574821.751069:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574821.751072:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788574821.751074:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574821.751084:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6734 00000100:00080000:1.0:1788574821.751092:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574821.751097:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574821.751106:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574821.751116:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c380 x1875449616455808/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574821.751132:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616455680:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788574821.751134:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574821.751142:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788574821.751144:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574821.751148:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6734 00000100:00080000:1.0:1788574821.751152:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574821.751156:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574821.751161:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574821.751171:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f100 x1875449616455936/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574821.751220:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574821.751257:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574821/real 1788574821] req@ffff9ada4566c700 x1875449616455680/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574846 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574821.751276:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c700 x1875449616455680/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574846 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574821.751287:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574821.751291:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574821.751295:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616455680:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574821.751321:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616455808:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574821.751345:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574821.751357:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574821/real 1788574821] req@ffff9ada4566c380 x1875449616455808/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574846 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574821.751370:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c380 x1875449616455808/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574846 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574821.751378:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574821.751382:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574821.751385:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616455808:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574821.751392:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616455936:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574821.751411:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574821.751422:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574821/real 1788574821] req@ffff9ada4566f100 x1875449616455936/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574846 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574821.751434:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f100 x1875449616455936/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574846 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574821.751441:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574821.751444:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574821.751447:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616455936:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574821.942963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574822.263020:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574822.966961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574823.286989:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574823.990960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574824.311056:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574825.014961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574825.334991:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574826.038959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574826.359017:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788574826.870958:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574826.870967:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788574826.870971:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574826.870981:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6739 00000100:00080000:1.0:1788574826.870992:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574826.870999:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574826.871035:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d180 x1875449616456064/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574826.871061:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574826.871065:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788574826.871067:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574826.871070:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6739 00000100:00080000:1.0:1788574826.871073:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574826.871076:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574826.871085:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574826.871095:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ca80 x1875449616456192/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574826.871104:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574826.871106:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788574826.871108:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574826.871111:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6739 00000100:00080000:1.0:1788574826.871113:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574826.871117:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574826.871121:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574826.871129:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d500 x1875449616456320/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574826.871237:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616456064:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788574826.871308:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574826.871343:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574826/real 1788574826] req@ffff9ada4566d180 x1875449616456064/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574851 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574826.871375:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d180 x1875449616456064/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574851 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574826.871384:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574826.871387:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574826.871390:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616456064:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574826.871396:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616456192:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574826.871416:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574826.871428:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574826/real 1788574826] req@ffff9ada4566ca80 x1875449616456192/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574851 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574826.871437:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ca80 x1875449616456192/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574851 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574826.871444:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574826.871448:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574826.871451:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616456192:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574826.871456:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616456320:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574826.871475:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574826.871483:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574826/real 1788574826] req@ffff9ada4566d500 x1875449616456320/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574851 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574826.871493:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d500 x1875449616456320/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574851 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574826.871500:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574826.871502:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574826.871505:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616456320:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574827.062964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574827.383008:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574828.086934:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574828.407079:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574829.110962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574829.431042:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574830.134965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574830.455044:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574831.158954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574831.479008:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788574831.991133:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574831.991143:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788574831.991147:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574831.991158:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6744 00000100:00080000:3.0:1788574831.991169:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574831.991177:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574831.991219:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3b80 x1875449616456448/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574831.991277:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574831.991281:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788574831.991283:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574831.991288:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6744 00000100:00080000:3.0:1788574831.991292:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574831.991296:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574831.991306:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574831.991320:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1c00 x1875449616456576/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788574831.991332:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616456448:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788574831.991333:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574831.991341:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788574831.991343:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574831.991347:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6744 00000100:00080000:3.0:1788574831.991352:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574831.991355:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574831.991360:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574831.991373:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0000 x1875449616456704/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788574831.991392:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574831.991437:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574831/real 1788574831] req@ffff9ada604e3b80 x1875449616456448/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574856 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574831.991456:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3b80 x1875449616456448/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574856 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574831.991466:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574831.991470:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574831.991474:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616456448:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574831.991496:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616456576:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574831.991513:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574831.991524:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574831/real 1788574831] req@ffff9ada604e1c00 x1875449616456576/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574856 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574831.991537:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1c00 x1875449616456576/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574856 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574831.991545:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574831.991548:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574831.991551:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616456576:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574831.991557:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616456704:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574831.991582:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574831.991591:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574831/real 1788574831] req@ffff9ada604e0000 x1875449616456704/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574856 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574831.991598:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0000 x1875449616456704/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574856 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574831.991607:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574831.991609:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574831.991611:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616456704:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574832.182990:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574832.502986:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574833.207008:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574833.527083:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574834.231022:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574834.551126:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574835.254996:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574835.575010:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574836.278946:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574836.599046:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574837.110981:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574837.110991:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574837.110995:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574837.111004:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6749 00000100:00080000:2.0:1788574837.111014:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574837.111020:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574837.111057:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627100 x1875449616456832/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574837.111083:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574837.111087:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574837.111089:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574837.111092:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6749 00000100:00080000:2.0:1788574837.111095:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574837.111108:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574837.111117:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574837.111130:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624700 x1875449616456960/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788574837.111138:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616456832:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788574837.111145:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574837.111149:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574837.111151:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000800:00000100:0.0:1788574837.111189:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00080000:2.0:1788574837.111195:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6749 00000100:00080000:2.0:1788574837.111201:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574837.111204:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574837.111209:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574837.111220:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574837/real 1788574837] req@ffff9ada60627100 x1875449616456832/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574862 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788574837.111225:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616457088/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788574837.111243:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627100 x1875449616456832/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574862 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574837.111251:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574837.111254:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574837.111259:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616456832:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574837.111276:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616456960:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574837.111289:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574837.111296:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574837/real 1788574837] req@ffff9ada60624700 x1875449616456960/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574862 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574837.111303:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624700 x1875449616456960/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574862 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574837.111313:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574837.111316:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574837.111320:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616456960:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574837.111325:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616457088:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574837.111343:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574837.111353:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574837/real 1788574837] req@ffff9ada60626680 x1875449616457088/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574862 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574837.111364:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616457088/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574862 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574837.111371:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574837.111373:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574837.111375:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616457088:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574837.302965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574837.622982:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574838.326952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574838.647026:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574839.350958:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574839.671029:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574840.374950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574840.695149:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574841.399008:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574841.719031:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574842.231026:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574842.231038:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574842.231042:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574842.231052:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6754 00000100:00080000:0.0:1788574842.231063:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574842.231071:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574842.231110:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616457216/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574842.231138:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574842.231142:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574842.231143:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574842.231147:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6754 00000100:00080000:0.0:1788574842.231151:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574842.231154:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574842.231163:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574842.231176:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616457344/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574842.231188:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1788574842.231189:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616457216:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788574842.231194:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574842.231199:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574842.231203:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6754 00000100:00080000:0.0:1788574842.231206:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574842.231210:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574842.231215:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574842.231228:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616457472/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574842.231264:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574842.231299:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574842/real 1788574842] req@ffff9ada4557d880 x1875449616457216/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574867 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574842.231316:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616457216/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574867 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574842.231326:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574842.231329:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574842.231333:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616457216:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574842.231396:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616457344:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574842.231415:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574842.231426:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574842/real 1788574842] req@ffff9ada4557f800 x1875449616457344/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574867 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574842.231437:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616457344/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574867 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574842.231444:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574842.231447:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574842.231450:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616457344:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574842.231455:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616457472:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574842.231470:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574842.231478:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574842/real 1788574842] req@ffff9ada4557ca80 x1875449616457472/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574867 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574842.231488:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616457472/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574867 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574842.231495:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574842.231498:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574842.231500:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616457472:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574842.422943:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574842.743153:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574843.446989:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574843.767057:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574844.471006:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574844.791022:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574845.494988:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574845.815053:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574846.519031:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574846.839093:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574847.351019:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574847.351030:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574847.351034:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574847.351047:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6759 00000100:00080000:2.0:1788574847.351060:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574847.351068:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574847.351105:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616457600/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574847.351131:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574847.351134:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574847.351135:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574847.351137:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6759 00000100:00080000:2.0:1788574847.351139:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574847.351141:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574847.351148:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574847.351155:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616457728/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574847.351162:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574847.351164:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574847.351166:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574847.351168:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6759 00000100:00080000:2.0:1788574847.351169:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574847.351172:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574847.351175:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574847.351181:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616457600:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788574847.351182:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616457856/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574847.351266:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616457728:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574847.351291:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616457856:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574847.542991:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574847.863098:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574848.566988:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574848.887003:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574849.590929:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574849.911050:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1788574850.423181:0:2373:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:2.0:1788574850.423228:0:2373:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1788574850.423244:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788574850.423296:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788574850.423306:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:3.0:1788574850.423417:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574847/real 1788574850] req@ffff9ada60624380 x1875449616457856/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574872 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574850.423444:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616457856/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574872 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574850.423457:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574850.423463:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574850.423468:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616457856:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574850.423476:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574847/real 1788574850] req@ffff9ada60626a00 x1875449616457600/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574872 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574850.423489:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616457600/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574872 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574850.423496:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574850.423500:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574850.423504:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616457600:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574850.423511:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574847/real 1788574850] req@ffff9ada60624a80 x1875449616457728/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574872 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574850.423523:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616457728/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574872 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574850.423538:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574850.423542:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574850.423545:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616457728:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574850.614975:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574850.935036:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574851.638932:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574851.959008:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788574852.471023:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574852.471044:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788574852.471049:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574852.471061:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6765 00000100:00080000:3.0:1788574852.471073:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574852.471082:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574852.471135:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2300 x1875449616457984/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574852.471184:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574852.471189:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788574852.471191:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574852.471196:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6765 00000100:00080000:3.0:1788574852.471201:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574852.471205:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574852.471218:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574852.471236:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0a80 x1875449616458112/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574852.471249:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574852.471254:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788574852.471256:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574852.471260:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6765 00000100:00080000:3.0:1788574852.471263:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1788574852.471270:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616457984:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788574852.471271:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574852.471281:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574852.471295:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2d80 x1875449616458240/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788574852.471395:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574852.471446:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574852/real 1788574852] req@ffff9ada604e2300 x1875449616457984/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574877 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574852.471484:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2300 x1875449616457984/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574877 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574852.471499:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574852.471504:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574852.471508:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616457984:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574852.471552:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616458112:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574852.471578:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574852.471592:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574852/real 1788574852] req@ffff9ada604e0a80 x1875449616458112/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574877 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574852.471605:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0a80 x1875449616458112/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574877 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574852.471614:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574852.471617:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574852.471620:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616458112:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574852.471626:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616458240:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574852.471648:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574852.471661:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574852/real 1788574852] req@ffff9ada604e2d80 x1875449616458240/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574877 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574852.471673:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2d80 x1875449616458240/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574877 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574852.471681:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574852.471684:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574852.471688:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616458240:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574852.663003:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574852.983038:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574853.686993:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574854.007004:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574854.711029:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574855.030981:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574855.734972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574856.055033:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574856.758966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574857.079089:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788574857.590982:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574857.590994:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788574857.590999:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574857.591010:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6770 00000100:00080000:3.0:1788574857.591020:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574857.591027:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574857.591059:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1880 x1875449616458368/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574857.591082:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574857.591085:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788574857.591086:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574857.591089:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6770 00000100:00080000:3.0:1788574857.591091:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574857.591093:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574857.591100:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574857.591110:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3800 x1875449616458496/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574857.591118:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574857.591120:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788574857.591122:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574857.591125:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6770 00000100:00080000:3.0:1788574857.591127:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1788574857.591130:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616458368:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788574857.591132:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574857.591140:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574857.591148:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1180 x1875449616458624/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788574857.591193:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574857.591275:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574857/real 1788574857] req@ffff9ada604e1880 x1875449616458368/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574882 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574857.591292:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1880 x1875449616458368/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574882 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574857.591307:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574857.591310:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574857.591313:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616458368:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574857.591319:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616458496:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574857.591338:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574857.591348:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574857/real 1788574857] req@ffff9ada604e3800 x1875449616458496/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574882 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574857.591356:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3800 x1875449616458496/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574882 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574857.591362:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574857.591364:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574857.591367:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616458496:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574857.591386:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616458624:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574857.591398:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574857.591404:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574857/real 1788574857] req@ffff9ada604e1180 x1875449616458624/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574882 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574857.591413:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1180 x1875449616458624/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574882 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574857.591418:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574857.591419:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574857.591421:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616458624:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574857.782938:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574858.103027:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574858.806921:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574859.126978:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574859.830940:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574860.151015:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574860.854955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574861.175048:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574861.879020:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574862.198988:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788574862.710970:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574862.710982:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788574862.710987:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574862.710999:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6775 00000100:00080000:3.0:1788574862.711009:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574862.711018:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574862.711057:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0700 x1875449616458752/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574862.711084:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574862.711087:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788574862.711088:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574862.711095:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6775 00000100:00080000:3.0:1788574862.711099:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574862.711102:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574862.711111:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574862.711120:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3480 x1875449616458880/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574862.711133:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574862.711136:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788574862.711137:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574862.711140:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6775 00000100:00080000:3.0:1788574862.711143:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1788574862.711146:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616458752:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788574862.711148:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574862.711155:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574862.711163:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0380 x1875449616459008/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788574862.711217:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574862.711252:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574862/real 1788574862] req@ffff9ada604e0700 x1875449616458752/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574887 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574862.711271:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0700 x1875449616458752/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574887 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574862.711282:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574862.711286:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574862.711290:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616458752:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574862.711296:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616458880:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574862.711318:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574862.711330:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574862/real 1788574862] req@ffff9ada604e3480 x1875449616458880/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574887 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574862.711341:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3480 x1875449616458880/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574887 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574862.711350:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574862.711353:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574862.711356:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616458880:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574862.711380:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616459008:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574862.711402:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574862.711414:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574862/real 1788574862] req@ffff9ada604e0380 x1875449616459008/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574887 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574862.711422:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0380 x1875449616459008/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574887 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574862.711432:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574862.711434:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574862.711436:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616459008:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574862.902957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574863.223064:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574863.926983:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574864.246986:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574864.950928:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574865.271058:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574865.975004:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574866.295003:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574866.998938:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574867.319088:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574867.830994:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574867.831007:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574867.831012:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574867.831024:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6780 00000100:00080000:0.0:1788574867.831037:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574867.831044:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574867.831092:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616459136/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574867.831124:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574867.831127:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574867.831129:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574867.831132:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6780 00000100:00080000:0.0:1788574867.831134:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574867.831137:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574867.831146:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574867.831156:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616459264/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574867.831165:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574867.831167:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574867.831169:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574867.831171:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6780 00000100:00080000:0.0:1788574867.831173:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574867.831176:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574867.831182:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574867.831182:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616459136:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574867.831190:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ce00 x1875449616459392/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574867.831271:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574867.831316:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574867/real 1788574867] req@ffff9ada4557f480 x1875449616459136/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574892 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574867.831369:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616459136/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574892 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574867.831382:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574867.831386:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574867.831391:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616459136:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574867.831399:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616459264:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574867.831426:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574867.831439:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574867/real 1788574867] req@ffff9ada4557c700 x1875449616459264/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574892 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574867.831453:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616459264/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574892 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574867.831462:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574867.831466:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574867.831470:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616459264:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574867.831500:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616459392:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574867.831523:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574867.831536:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574867/real 1788574867] req@ffff9ada4557ce00 x1875449616459392/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574892 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574867.831560:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ce00 x1875449616459392/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574892 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574867.831569:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574867.831573:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574867.831577:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616459392:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574868.022994:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574868.342976:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574869.046958:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574869.367112:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574870.070941:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574870.391022:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574871.094983:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574871.415358:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574872.118995:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574872.439094:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574872.950974:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574872.950988:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574872.950993:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574872.951007:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6785 00000100:00080000:0.0:1788574872.951021:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574872.951029:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574872.951074:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c000 x1875449616459520/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574872.951113:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574872.951118:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574872.951121:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574872.951126:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6785 00000100:00080000:0.0:1788574872.951130:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574872.951135:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574872.951148:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574872.951163:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c380 x1875449616459648/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574872.951171:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616459520:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788574872.951179:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574872.951185:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574872.951187:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574872.951193:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6785 00000100:00080000:0.0:1788574872.951196:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574872.951200:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574872.951205:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574872.951231:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ce00 x1875449616459776/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574872.951266:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574872.951313:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574872/real 1788574872] req@ffff9ada4557c000 x1875449616459520/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574897 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574872.951348:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c000 x1875449616459520/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574897 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574872.951357:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574872.951360:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574872.951363:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616459520:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574872.951388:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616459648:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574872.951406:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574872.951416:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574872/real 1788574872] req@ffff9ada4557c380 x1875449616459648/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574897 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574872.951425:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c380 x1875449616459648/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574897 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574872.951431:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574872.951441:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574872.951444:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616459648:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574872.951451:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616459776:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574872.951464:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574872.951471:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574872/real 1788574872] req@ffff9ada4557ce00 x1875449616459776/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574897 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574872.951487:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ce00 x1875449616459776/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574897 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574872.951506:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574872.951510:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574872.951513:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616459776:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574873.143019:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574873.463038:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574874.166984:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574874.487007:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574875.190984:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574875.511067:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574876.214968:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574876.535068:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574877.238973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574877.559051:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574878.070928:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574878.070939:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574878.070943:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574878.070952:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6790 00000100:00080000:0.0:1788574878.070965:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574878.070972:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574878.071012:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616459904/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574878.071036:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574878.071039:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574878.071040:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574878.071043:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6790 00000100:00080000:0.0:1788574878.071045:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574878.071047:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574878.071056:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574878.071063:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616460032/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574878.071070:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574878.071072:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574878.071074:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574878.071076:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6790 00000100:00080000:0.0:1788574878.071077:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574878.071079:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574878.071095:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574878.071097:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616459904:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574878.071105:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616460160/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574878.071164:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574878.071196:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574878/real 1788574878] req@ffff9ada4557c700 x1875449616459904/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574903 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574878.071210:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616459904/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574903 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574878.071218:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574878.071221:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574878.071224:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616459904:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574878.071229:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616460032:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574878.071249:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574878.071257:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574878/real 1788574878] req@ffff9ada4557f480 x1875449616460032/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574903 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574878.071264:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616460032/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574903 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574878.071269:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574878.071271:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574878.071273:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616460032:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574878.071295:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616460160:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574878.071306:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574878.071311:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574878/real 1788574878] req@ffff9ada4557ca80 x1875449616460160/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574903 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574878.071318:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616460160/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574903 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574878.071323:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574878.071324:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574878.071326:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616460160:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574878.262992:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574878.583175:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574879.286958:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574879.607015:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574880.310987:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574880.631057:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574881.334969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574881.655027:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574882.358953:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574882.679119:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574883.190946:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574883.190953:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574883.190957:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574883.190965:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6795 00000100:00080000:2.0:1788574883.190972:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574883.190982:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574883.191011:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616460288/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574883.191030:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574883.191032:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574883.191033:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574883.191035:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6795 00000100:00080000:2.0:1788574883.191037:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574883.191039:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574883.191057:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574883.191067:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616460416/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574883.191075:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574883.191077:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574883.191078:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574883.191081:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6795 00000100:00080000:2.0:1788574883.191083:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574883.191086:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574883.191089:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574883.191094:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616460288:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788574883.191097:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616460544/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574883.191159:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574883.191190:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574883/real 1788574883] req@ffff9ada60625880 x1875449616460288/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574908 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574883.191208:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616460288/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574908 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574883.191217:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574883.191221:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574883.191225:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616460288:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574883.191232:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616460416:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574883.191255:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574883.191267:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574883/real 1788574883] req@ffff9ada60625500 x1875449616460416/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574908 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574883.191279:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616460416/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574908 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574883.191287:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574883.191290:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574883.191293:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616460416:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574883.191317:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616460544:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574883.191336:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574883.191346:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574883/real 1788574883] req@ffff9ada60625c00 x1875449616460544/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574908 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574883.191357:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616460544/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574908 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574883.191365:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574883.191369:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574883.191371:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616460544:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574883.382982:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574883.702979:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574884.406960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574884.727117:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574885.430988:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574885.751113:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574886.455001:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574886.775057:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574887.478964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574887.799061:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574888.310989:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574888.311001:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574888.311036:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574888.311047:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6800 00000100:00080000:2.0:1788574888.311059:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574888.311067:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574888.311105:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616460672/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574888.311138:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574888.311143:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574888.311145:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574888.311149:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6800 00000100:00080000:2.0:1788574888.311152:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574888.311156:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574888.311166:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574888.311180:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616460800/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574888.311193:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574888.311197:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574888.311202:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1788574888.311203:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616460672:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788574888.311210:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6800 00000100:00080000:2.0:1788574888.311215:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574888.311220:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574888.311226:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574888.311242:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616460928/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574888.311264:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574888.311297:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574888/real 1788574888] req@ffff9ada60625f80 x1875449616460672/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574913 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574888.311312:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616460672/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574913 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574888.311319:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574888.311322:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574888.311325:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616460672:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574888.311349:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616460800:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574888.311373:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574888.311386:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574888/real 1788574888] req@ffff9ada60624e00 x1875449616460800/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574913 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574888.311395:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616460800/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574913 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574888.311401:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574888.311403:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574888.311406:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616460800:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574888.311410:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616460928:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574888.311423:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574888.311430:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574888/real 1788574888] req@ffff9ada60627480 x1875449616460928/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574913 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574888.311437:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616460928/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574913 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574888.311447:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574888.311449:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574888.311451:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616460928:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574888.502964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574888.823070:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574889.526960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574889.847037:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574890.550959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574890.871019:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574891.574966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574891.895003:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574892.598953:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574892.919264:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574893.430995:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574893.431004:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574893.431007:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574893.431016:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6805 00000100:00080000:2.0:1788574893.431025:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574893.431031:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574893.431063:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616461056/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574893.431088:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574893.431091:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574893.431093:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574893.431095:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6805 00000100:00080000:2.0:1788574893.431098:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574893.431100:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574893.431107:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574893.431114:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616461184/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574893.431128:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574893.431131:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574893.431133:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574893.431136:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6805 00000100:00080000:2.0:1788574893.431138:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574893.431140:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574893.431143:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574893.431151:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616461056:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788574893.431152:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616461312/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574893.431218:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574893.431254:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574893/real 1788574893] req@ffff9ada60625180 x1875449616461056/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574918 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574893.431275:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616461056/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574918 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574893.431286:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574893.431291:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574893.431309:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616461056:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574893.431317:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616461184:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574893.431339:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574893.431349:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574893/real 1788574893] req@ffff9ada60626d80 x1875449616461184/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574918 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574893.431357:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616461184/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574918 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574893.431367:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574893.431370:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574893.431372:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616461184:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574893.431390:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616461312:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574893.431401:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574893.431407:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574893/real 1788574893] req@ffff9ada60627b80 x1875449616461312/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574918 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574893.431415:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616461312/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574918 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574893.431420:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574893.431422:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574893.431424:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616461312:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574893.623008:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574893.943016:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574894.646985:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574894.967056:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574895.671003:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574895.991098:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574896.694992:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574897.015017:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574897.718964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574898.039017:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574898.550993:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574898.551003:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574898.551007:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574898.551018:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6811 00000100:00080000:2.0:1788574898.551029:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574898.551036:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574898.551070:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627800 x1875449616461440/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574898.551102:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574898.551106:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574898.551108:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574898.551111:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6811 00000100:00080000:2.0:1788574898.551116:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574898.551119:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574898.551128:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574898.551140:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626300 x1875449616461568/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574898.551150:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574898.551155:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574898.551158:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1788574898.551158:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616461440:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788574898.551166:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6811 00000100:00080000:2.0:1788574898.551170:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574898.551174:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574898.551178:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574898.551191:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616461696/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574898.551227:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574898.551257:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574898/real 1788574898] req@ffff9ada60627800 x1875449616461440/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574923 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574898.551272:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627800 x1875449616461440/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574923 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574898.551284:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574898.551287:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574898.551290:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616461440:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574898.551309:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616461568:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574898.551325:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574898.551333:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574898/real 1788574898] req@ffff9ada60626300 x1875449616461568/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574923 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574898.551341:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626300 x1875449616461568/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574923 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574898.551347:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574898.551349:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574898.551351:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616461568:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574898.551355:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616461696:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574898.551365:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574898.551370:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574898/real 1788574898] req@ffff9ada60627b80 x1875449616461696/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574923 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574898.551378:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616461696/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574923 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574898.551383:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574898.551384:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574898.551386:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616461696:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574898.742976:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574899.063020:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574899.766972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574900.087206:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574900.790977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574901.110996:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574901.814951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574902.135044:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574902.838980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574903.159030:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574903.670994:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574903.671003:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574903.671007:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574903.671015:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6816 00000100:00080000:2.0:1788574903.671024:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574903.671030:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574903.671074:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616461824/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574903.671098:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574903.671101:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574903.671102:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574903.671105:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6816 00000100:00080000:2.0:1788574903.671107:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574903.671109:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574903.671116:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574903.671125:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616461952/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574903.671132:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574903.671135:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574903.671136:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574903.671139:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6816 00000100:00080000:2.0:1788574903.671142:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574903.671144:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574903.671147:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574903.671150:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616461824:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788574903.671156:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616462080/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574903.671207:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574903.671235:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574903/real 1788574903] req@ffff9ada60626d80 x1875449616461824/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574928 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574903.671251:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616461824/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574928 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574903.671262:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574903.671266:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574903.671270:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616461824:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574903.671278:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616461952:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574903.671300:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574903.671310:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574903/real 1788574903] req@ffff9ada60625180 x1875449616461952/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574928 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574903.671318:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616461952/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574928 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574903.671324:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574903.671326:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574903.671337:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616461952:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574903.671359:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616462080:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574903.671373:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574903.671380:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574903/real 1788574903] req@ffff9ada60627480 x1875449616462080/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574928 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574903.671388:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616462080/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574928 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574903.671393:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574903.671395:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574903.671397:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616462080:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574903.862942:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574904.183042:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574904.886995:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574905.207051:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574905.910971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574906.231085:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574906.934971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574907.255414:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574907.958990:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574908.279082:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574908.790992:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574908.791004:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574908.791009:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574908.791020:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6821 00000100:00080000:2.0:1788574908.791032:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574908.791040:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574908.791096:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616462208/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574908.791131:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574908.791136:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574908.791138:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574908.791143:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6821 00000100:00080000:2.0:1788574908.791147:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574908.791153:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574908.791166:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574908.791183:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616462336/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574908.791187:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616462208:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788574908.791201:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574908.791208:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574908.791211:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574908.791216:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6821 00000100:00080000:2.0:1788574908.791220:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574908.791225:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574908.791230:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574908.791245:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616462464/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574908.791292:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574908.791331:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574908/real 1788574908] req@ffff9ada60624e00 x1875449616462208/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574933 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574908.791349:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616462208/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574933 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574908.791358:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574908.791362:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574908.791365:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616462208:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574908.791390:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616462336:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574908.791421:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574908.791436:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574908/real 1788574908] req@ffff9ada60625f80 x1875449616462336/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574933 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574908.791449:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616462336/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574933 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574908.791459:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574908.791471:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574908.791476:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616462336:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574908.791482:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616462464:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574908.791501:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574908.791510:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574908/real 1788574908] req@ffff9ada60625c00 x1875449616462464/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574933 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574908.791519:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616462464/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574933 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574908.791525:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574908.791527:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574908.791529:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616462464:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574908.983009:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574909.303117:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574910.006961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574910.327070:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574911.030971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574911.351052:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574912.054979:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574912.375055:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574913.079108:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574913.399036:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574913.911025:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574913.911041:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574913.911047:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574913.911061:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6826 00000100:00080000:0.0:1788574913.911076:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574913.911086:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574913.911142:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616462592/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574913.911187:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574913.911192:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574913.911194:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574913.911199:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6826 00000100:00080000:0.0:1788574913.911204:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574913.911208:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574913.911222:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574913.911242:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616462720/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574913.911261:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574913.911266:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574913.911268:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574913.911275:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6826 00000100:00080000:0.0:1788574913.911284:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1788574913.911286:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616462592:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788574913.911292:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574913.911306:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574913.911325:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616462848/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574913.911428:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616462720:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574913.911493:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616462848:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788574914.102972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574914.423090:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574915.127030:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574915.447050:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788574916.150930:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574916.471018:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1788574916.983128:0:2371:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:0.0:1788574916.983165:0:2371:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1788574916.983179:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:0.0:1788574916.983230:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:0.0:1788574916.983240:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:3.0:1788574916.983348:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574913/real 1788574916] req@ffff9ada4557dc00 x1875449616462848/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574938 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574916.983393:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616462848/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574938 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574916.983407:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574916.983414:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574916.983418:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616462848:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574916.983433:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574913/real 1788574916] req@ffff9ada4557f800 x1875449616462592/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574938 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574916.983447:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616462592/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574938 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574916.983455:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574916.983458:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574916.983462:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616462592:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574916.983469:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574913/real 1788574916] req@ffff9ada4557d880 x1875449616462720/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574938 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574916.983482:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616462720/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574938 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574916.983496:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574916.983500:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574916.983504:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616462720:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574917.174991:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574917.495070:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574918.198944:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574918.519049:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574919.031028:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574919.031038:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574919.031041:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574919.031050:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6831 00000100:00080000:0.0:1788574919.031059:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574919.031065:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574919.031097:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616462976/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574919.031131:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574919.031135:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574919.031137:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574919.031141:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6831 00000100:00080000:0.0:1788574919.031145:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574919.031149:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574919.031160:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574919.031182:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616463104/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574919.031189:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616462976:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788574919.031200:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574919.031205:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574919.031207:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574919.031211:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6831 00000100:00080000:0.0:1788574919.031215:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574919.031220:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574919.031226:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574919.031242:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616463232/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574919.031287:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574919.031323:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574919/real 1788574919] req@ffff9ada4557e300 x1875449616462976/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574944 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574919.031343:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616462976/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574944 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574919.031354:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574919.031359:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574919.031363:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616462976:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574919.031390:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616463104:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574919.031414:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574919.031427:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574919/real 1788574919] req@ffff9ada4557fb80 x1875449616463104/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574944 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574919.031439:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616463104/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574944 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574919.031452:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574919.031456:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574919.031459:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616463104:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574919.031466:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616463232:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574919.031486:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574919.031498:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574919/real 1788574919] req@ffff9ada4557ed80 x1875449616463232/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574944 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574919.031511:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616463232/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574944 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574919.031518:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574919.031522:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574919.031526:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616463232:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574919.222954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574919.543017:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574920.246970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574920.566985:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574921.270945:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574921.591039:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574922.294979:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574922.614989:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574923.318944:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574923.639021:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788574924.150954:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574924.150962:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788574924.150967:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574924.150975:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6836 00000100:00080000:3.0:1788574924.150984:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574924.150989:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574924.151023:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0380 x1875449616463360/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574924.151048:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788574924.151051:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788574924.151052:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574924.151055:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6836 00000100:00080000:3.0:1788574924.151058:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574924.151060:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574924.151067:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574924.151076:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3480 x1875449616463488/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788574924.151084:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1788574924.151089:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616463360:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788574924.151089:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788574924.151093:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788574924.151101:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6836 00000100:00080000:3.0:1788574924.151103:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788574924.151105:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788574924.151109:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574924.151118:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0700 x1875449616463616/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788574924.151164:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574924.151193:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574924/real 1788574924] req@ffff9ada604e0380 x1875449616463360/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574949 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574924.151207:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0380 x1875449616463360/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574949 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574924.151215:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574924.151218:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574924.151221:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616463360:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788574924.151245:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616463488:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574924.151263:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574924.151276:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574924/real 1788574924] req@ffff9ada604e3480 x1875449616463488/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574949 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574924.151289:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3480 x1875449616463488/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574949 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574924.151299:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574924.151303:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574924.151306:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616463488:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788574924.151313:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616463616:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788574924.151333:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788574924.151350:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574924/real 1788574924] req@ffff9ada604e0700 x1875449616463616/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574949 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788574924.151359:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0700 x1875449616463616/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574949 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788574924.151365:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788574924.151367:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788574924.151369:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616463616:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574924.342991:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574924.663048:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574925.367011:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574925.687061:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574926.390930:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574926.711013:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574927.414974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574927.735052:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574928.438967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574928.759091:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574929.270949:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574929.270958:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574929.270967:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574929.270975:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6841 00000100:00080000:0.0:1788574929.270984:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574929.270991:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574929.271024:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616463744/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574929.271047:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574929.271050:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574929.271052:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574929.271055:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6841 00000100:00080000:0.0:1788574929.271058:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574929.271061:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574929.271068:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574929.271081:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616463872/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574929.271082:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616463744:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788574929.271096:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574929.271099:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574929.271100:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574929.271104:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6841 00000100:00080000:0.0:1788574929.271106:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574929.271108:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574929.271112:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574929.271122:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557df80 x1875449616464000/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574929.271146:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574929.271175:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574929/real 1788574929] req@ffff9ada4557ea00 x1875449616463744/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574954 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574929.271189:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616463744/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574954 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574929.271197:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574929.271200:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574929.271203:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616463744:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574929.271221:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616463872:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574929.271234:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574929.271253:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574929/real 1788574929] req@ffff9ada4557e680 x1875449616463872/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574954 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574929.271262:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616463872/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574954 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574929.271268:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574929.271270:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574929.271273:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616463872:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574929.271277:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616464000:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574929.271288:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574929.271294:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574929/real 1788574929] req@ffff9ada4557df80 x1875449616464000/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574954 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574929.271301:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557df80 x1875449616464000/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574954 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574929.271306:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574929.271308:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574929.271310:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616464000:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574929.462984:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574929.783035:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574930.486956:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574930.806988:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574931.510928:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574931.831021:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574932.534929:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574932.854985:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574933.558939:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574933.879020:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574934.390980:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574934.390991:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574934.390995:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574934.391006:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6846 00000100:00080000:2.0:1788574934.391017:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574934.391024:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574934.391059:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616464128/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574934.391082:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574934.391085:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574934.391086:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574934.391089:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6846 00000100:00080000:2.0:1788574934.391092:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574934.391094:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574934.391102:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574934.391116:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616464256/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574934.391129:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574934.391135:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1788574934.391135:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616464128:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788574934.391138:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574934.391147:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6846 00000100:00080000:2.0:1788574934.391151:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574934.391155:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574934.391160:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574934.391171:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616464384/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574934.391198:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574934.391229:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574934/real 1788574934] req@ffff9ada60625500 x1875449616464128/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574959 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574934.391244:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616464128/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574959 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574934.391257:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574934.391260:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574934.391263:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616464128:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574934.391281:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616464256:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574934.391294:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574934.391302:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574934/real 1788574934] req@ffff9ada60625880 x1875449616464256/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574959 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574934.391310:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616464256/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574959 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574934.391316:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574934.391318:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574934.391320:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616464256:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574934.391324:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616464384:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574934.391333:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574934.391338:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574934/real 1788574934] req@ffff9ada60624a80 x1875449616464384/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574959 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574934.391345:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616464384/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574959 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574934.391351:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574934.391353:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574934.391354:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616464384:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574934.582970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574934.903031:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574935.606945:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574935.927133:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574936.630965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574936.951062:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574937.654982:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574937.975020:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574938.678956:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574938.999081:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574939.510920:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574939.510928:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574939.510931:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574939.510938:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6852 00000100:00080000:2.0:1788574939.510946:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574939.510951:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574939.510980:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616464512/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574939.511002:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574939.511004:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574939.511005:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574939.511008:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6852 00000100:00080000:2.0:1788574939.511010:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574939.511012:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574939.511019:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574939.511026:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616464640/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574939.511033:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574939.511036:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574939.511037:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574939.511039:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6852 00000100:00080000:2.0:1788574939.511041:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574939.511042:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574939.511045:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574939.511049:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616464512:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788574939.511054:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616464768/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574939.511105:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574939.511132:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574939/real 1788574939] req@ffff9ada60626a00 x1875449616464512/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574964 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574939.511145:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616464512/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574964 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574939.511152:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574939.511155:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574939.511157:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616464512:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574939.511161:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616464640:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574939.511173:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574939.511180:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574939/real 1788574939] req@ffff9ada60624380 x1875449616464640/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574964 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574939.511187:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616464640/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574964 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574939.511191:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574939.511193:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574939.511195:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616464640:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574939.511212:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616464768:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574939.511221:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574939.511226:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574939/real 1788574939] req@ffff9ada60626680 x1875449616464768/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574964 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574939.511233:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616464768/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574964 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574939.511237:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574939.511238:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574939.511248:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616464768:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574939.703004:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574940.023018:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574940.726958:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574941.047000:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574941.750959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574942.071007:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574942.774946:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574943.095004:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574943.798956:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574944.119020:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574944.630983:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574944.630992:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574944.630997:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574944.631006:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6857 00000100:00080000:2.0:1788574944.631016:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574944.631022:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574944.631058:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624700 x1875449616464896/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574944.631083:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574944.631087:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574944.631089:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574944.631092:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6857 00000100:00080000:2.0:1788574944.631096:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574944.631108:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574944.631121:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574944.631122:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616464896:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788574944.631144:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627100 x1875449616465024/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574944.631155:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574944.631158:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574944.631160:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574944.631164:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6857 00000100:00080000:2.0:1788574944.631169:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574944.631172:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574944.631178:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1788574944.631178:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788574944.631192:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616465152/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574944.631212:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574944/real 1788574944] req@ffff9ada60624700 x1875449616464896/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574969 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574944.631231:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624700 x1875449616464896/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574969 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574944.631248:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574944.631251:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574944.631255:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616464896:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574944.631272:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616465024:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574944.631296:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574944.631308:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574944/real 1788574944] req@ffff9ada60627100 x1875449616465024/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574969 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574944.631319:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627100 x1875449616465024/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574969 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574944.631327:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574944.631330:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574944.631333:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616465024:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574944.631339:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616465152:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574944.631364:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574944.631372:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574944/real 1788574944] req@ffff9ada60626680 x1875449616465152/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574969 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574944.631379:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616465152/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574969 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574944.631384:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574944.631386:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574944.631388:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616465152:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574944.822936:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574945.142974:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574945.846962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574946.166990:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574946.870970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574947.191007:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574947.894972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574948.214988:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574948.918954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574949.239033:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574949.751007:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574949.751014:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574949.751017:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574949.751025:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6862 00000100:00080000:0.0:1788574949.751034:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574949.751040:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574949.751072:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d180 x1875449616465280/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574949.751091:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574949.751093:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574949.751095:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574949.751097:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6862 00000100:00080000:0.0:1788574949.751099:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574949.751102:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574949.751108:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574949.751116:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d500 x1875449616465408/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574949.751125:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574949.751127:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574949.751129:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1788574949.751133:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616465280:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788574949.751134:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6862 00000100:00080000:0.0:1788574949.751138:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574949.751142:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574949.751145:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574949.751154:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557df80 x1875449616465536/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574949.751188:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574949.751215:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574949/real 1788574949] req@ffff9ada4557d180 x1875449616465280/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574974 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574949.751227:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d180 x1875449616465280/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574974 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574949.751234:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574949.751237:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574949.751240:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616465280:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574949.751255:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616465408:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574949.751268:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574949.751275:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574949/real 1788574949] req@ffff9ada4557d500 x1875449616465408/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574974 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574949.751282:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d500 x1875449616465408/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574974 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574949.751287:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574949.751289:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574949.751291:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616465408:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574949.751295:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616465536:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574949.751304:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574949.751309:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574949/real 1788574949] req@ffff9ada4557df80 x1875449616465536/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574974 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574949.751315:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557df80 x1875449616465536/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574974 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574949.751319:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574949.751321:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574949.751322:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616465536:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574949.942984:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574950.263054:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574950.966967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788574951.287049:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574951.990967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574952.311026:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574953.014942:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574953.335020:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574954.038969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574954.358993:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574954.870976:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574954.870986:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574954.870990:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574954.871000:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6867 00000100:00080000:0.0:1788574954.871010:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574954.871017:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574954.871051:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616465664/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574954.871079:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574954.871083:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574954.871086:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574954.871089:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6867 00000100:00080000:0.0:1788574954.871093:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574954.871096:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574954.871105:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574954.871128:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616465792/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574954.871139:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574954.871143:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574954.871145:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574954.871149:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6867 00000100:00080000:0.0:1788574954.871153:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574954.871158:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788574954.871159:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616465664:192.168.202.113@tcp:250:kworker.0 00000008:00100000:0.0:1788574954.871164:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574954.871177:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616465920/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574954.871217:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574954.871242:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574954/real 1788574954] req@ffff9ada4557e680 x1875449616465664/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574979 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574954.871258:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616465664/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574979 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574954.871265:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574954.871267:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574954.871270:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616465664:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574954.871274:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616465792:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574954.871287:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574954.871293:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574954/real 1788574954] req@ffff9ada4557ea00 x1875449616465792/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574979 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574954.871299:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616465792/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574979 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574954.871304:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574954.871305:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574954.871307:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616465792:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574954.871322:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616465920:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574954.871334:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574954.871343:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574954/real 1788574954] req@ffff9ada4557ed80 x1875449616465920/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574979 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574954.871355:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616465920/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574979 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574954.871380:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574954.871382:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574954.871384:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616465920:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574955.062926:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574955.383006:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574956.087032:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574956.407027:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574957.110946:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574957.431024:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574958.135002:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574958.455052:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574959.158986:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574959.479062:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574959.991007:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574959.991018:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574959.991022:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574959.991032:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6872 00000100:00080000:0.0:1788574959.991044:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574959.991050:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574959.991093:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616466048/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574959.991133:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574959.991138:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574959.991140:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574959.991145:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6872 00000100:00080000:0.0:1788574959.991148:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574959.991152:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574959.991163:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574959.991176:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616466176/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574959.991186:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574959.991189:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574959.991191:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574959.991194:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6872 00000100:00080000:0.0:1788574959.991196:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574959.991198:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574959.991202:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574959.991211:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616466304/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574959.991217:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616466048:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788574959.991295:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574959.991327:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574959/real 1788574959] req@ffff9ada4557fb80 x1875449616466048/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574984 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574959.991355:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616466048/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574984 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574959.991363:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574959.991366:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574959.991369:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616466048:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574959.991374:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616466176:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574959.991390:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574959.991398:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574959/real 1788574959] req@ffff9ada4557e300 x1875449616466176/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574984 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574959.991405:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616466176/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574984 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574959.991409:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574959.991411:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574959.991413:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616466176:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574959.991430:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616466304:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574959.991440:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574959.991445:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574959/real 1788574959] req@ffff9ada4557d880 x1875449616466304/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574984 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574959.991452:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616466304/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574984 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574959.991456:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574959.991458:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574959.991459:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616466304:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574960.182948:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574960.503074:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574961.206955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574961.527098:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574962.230923:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574962.550990:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574963.254992:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574963.575058:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574964.278960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574964.599079:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574965.110925:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574965.110933:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574965.110936:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574965.110943:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6877 00000100:00080000:0.0:1788574965.110951:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574965.110957:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574965.110988:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616466432/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574965.111008:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574965.111011:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574965.111012:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574965.111014:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6877 00000100:00080000:0.0:1788574965.111016:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574965.111018:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574965.111024:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574965.111032:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616466560/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574965.111038:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574965.111040:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574965.111042:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574965.111044:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6877 00000100:00080000:0.0:1788574965.111046:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574965.111048:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574965.111051:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574965.111059:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616466688/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574965.111061:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616466432:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788574965.111121:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574965.111154:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574965/real 1788574965] req@ffff9ada4557f800 x1875449616466432/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574990 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574965.111172:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616466432/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574990 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574965.111183:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574965.111187:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574965.111190:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616466432:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574965.111196:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616466560:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574965.111215:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574965.111226:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574965/real 1788574965] req@ffff9ada4557dc00 x1875449616466560/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574990 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574965.111237:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616466560/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574990 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574965.111252:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574965.111255:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574965.111259:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616466560:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574965.111281:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616466688:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574965.111299:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574965.111308:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574965/real 1788574965] req@ffff9ada4557ca80 x1875449616466688/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574990 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574965.111330:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616466688/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574990 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574965.111337:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574965.111340:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574965.111343:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616466688:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574965.302964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574965.623011:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574966.326964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574966.647029:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574967.350990:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574967.671009:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574968.374994:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574968.695051:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574969.398966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574969.719004:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574970.230967:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574970.230974:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574970.230978:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574970.230985:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6882 00000100:00080000:0.0:1788574970.230993:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574970.230999:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574970.231029:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616466816/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574970.231052:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574970.231055:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574970.231056:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574970.231058:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6882 00000100:00080000:0.0:1788574970.231060:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574970.231062:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574970.231069:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574970.231077:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616466944/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574970.231084:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574970.231086:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574970.231088:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574970.231091:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6882 00000100:00080000:0.0:1788574970.231093:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1788574970.231093:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616466816:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788574970.231098:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574970.231103:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574970.231109:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ce00 x1875449616467072/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574970.231145:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574970.231172:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574970/real 1788574970] req@ffff9ada4557f480 x1875449616466816/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574995 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574970.231185:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616466816/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788574995 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574970.231191:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574970.231194:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574970.231196:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616466816:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574970.231214:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616466944:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574970.231227:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574970.231234:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574970/real 1788574970] req@ffff9ada4557c700 x1875449616466944/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574995 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574970.231241:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616466944/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574995 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574970.231246:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574970.231248:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574970.231250:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616466944:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574970.231253:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616467072:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574970.231261:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574970.231266:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574970/real 1788574970] req@ffff9ada4557ce00 x1875449616467072/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574995 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574970.231272:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ce00 x1875449616467072/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788574995 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574970.231277:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574970.231278:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574970.231280:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616467072:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574970.422955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574970.743034:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574971.446994:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574971.767058:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574972.471010:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574972.791150:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574973.494968:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574973.815043:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574974.518926:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574974.839034:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788574975.350969:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574975.350977:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788574975.350981:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574975.350988:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6887 00000100:00080000:0.0:1788574975.350997:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574975.351023:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788574975.351059:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c380 x1875449616467200/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574975.351083:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574975.351085:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788574975.351087:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574975.351089:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6887 00000100:00080000:0.0:1788574975.351092:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574975.351094:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574975.351101:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574975.351108:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c000 x1875449616467328/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788574975.351115:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788574975.351117:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788574975.351118:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788574975.351121:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6887 00000100:00080000:0.0:1788574975.351123:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788574975.351125:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788574975.351129:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788574975.351136:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ce00 x1875449616467456/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574975.351136:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616467200:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788574975.351192:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574975.351219:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574975/real 1788574975] req@ffff9ada4557c380 x1875449616467200/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575000 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574975.351232:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c380 x1875449616467200/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575000 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574975.351239:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574975.351242:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574975.351245:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616467200:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574975.351249:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616467328:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574975.351262:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574975.351268:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574975/real 1788574975] req@ffff9ada4557c000 x1875449616467328/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575000 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574975.351275:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c000 x1875449616467328/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575000 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574975.351280:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574975.351281:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574975.351283:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616467328:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574975.351300:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616467456:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574975.351311:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574975.351316:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574975/real 1788574975] req@ffff9ada4557ce00 x1875449616467456/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575000 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574975.351322:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ce00 x1875449616467456/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575000 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574975.351327:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574975.351328:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574975.351330:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616467456:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574975.542966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574975.863037:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574976.566998:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574976.886980:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574977.591009:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574977.911044:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574978.614987:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574978.935035:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574979.638970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574979.959072:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788574980.470964:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574980.470982:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788574980.470987:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574980.470999:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6893 00000100:00080000:1.0:1788574980.471012:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574980.471021:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788574980.471076:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f480 x1875449616467584/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574980.471108:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574980.471111:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788574980.471113:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574980.471116:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6893 00000100:00080000:1.0:1788574980.471119:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574980.471122:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574980.471132:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788574980.471143:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e300 x1875449616467712/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788574980.471150:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788574980.471154:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788574980.471155:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788574980.471157:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6893 00000100:00080000:1.0:1788574980.471159:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788574980.471162:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788574980.471167:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788574980.471169:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616467584:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788574980.471177:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ea00 x1875449616467840/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574980.471322:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616467712:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574980.471360:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616467840:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574980.662995:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574980.983166:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574981.687005:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574982.007115:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574982.710990:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788574983.031056:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1788574983.543112:0:2370:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:2.0:1788574983.543154:0:2370:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1788574983.543169:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788574983.543213:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788574983.543224:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:3.0:1788574983.543269:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574980/real 1788574983] req@ffff9ada4566ea00 x1875449616467840/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575005 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574983.543299:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ea00 x1875449616467840/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575005 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574983.543312:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574983.543317:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574983.543322:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616467840:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574983.543331:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574980/real 1788574983] req@ffff9ada4566f480 x1875449616467584/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575005 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574983.543346:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f480 x1875449616467584/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575005 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574983.543355:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574983.543392:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574983.543397:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616467584:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574983.543405:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574980/real 1788574983] req@ffff9ada4566e300 x1875449616467712/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575005 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574983.543420:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e300 x1875449616467712/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575005 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574983.543428:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574983.543433:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574983.543437:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616467712:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574983.734954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574984.055005:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574984.759014:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574985.079035:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574985.591017:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574985.591024:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574985.591027:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574985.591036:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6898 00000100:00080000:2.0:1788574985.591044:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574985.591050:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574985.591078:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616467968/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574985.591111:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574985.591115:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574985.591117:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574985.591121:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6898 00000100:00080000:2.0:1788574985.591125:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574985.591129:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574985.591138:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574985.591152:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616468096/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574985.591164:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574985.591168:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574985.591170:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574985.591174:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6898 00000100:00080000:2.0:1788574985.591180:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1788574985.591180:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616467968:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788574985.591187:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574985.591195:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574985.591204:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616468224/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788574985.591262:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574985.591301:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574985/real 1788574985] req@ffff9ada60624380 x1875449616467968/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575010 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574985.591328:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616467968/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575010 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574985.591340:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574985.591344:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574985.591348:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616467968:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574985.591373:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616468096:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574985.591399:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574985.591412:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574985/real 1788574985] req@ffff9ada60626a00 x1875449616468096/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575010 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574985.591424:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616468096/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575010 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574985.591432:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574985.591436:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574985.591439:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616468096:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574985.591445:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616468224:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574985.591500:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574985.591513:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574985/real 1788574985] req@ffff9ada60624a80 x1875449616468224/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575010 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574985.591523:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616468224/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575010 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574985.591532:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574985.591535:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574985.591538:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616468224:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574985.782974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574986.103109:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574986.806986:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574987.127073:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574987.830970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574988.151092:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574988.854962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574989.175067:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574989.878994:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574990.199064:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574990.710960:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574990.710968:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574990.710972:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574990.710987:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6903 00000100:00080000:2.0:1788574990.710997:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574990.711003:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574990.711039:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616468352/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574990.711063:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574990.711065:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574990.711067:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574990.711069:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6903 00000100:00080000:2.0:1788574990.711072:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574990.711074:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574990.711082:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574990.711091:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616468480/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574990.711098:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574990.711100:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574990.711102:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574990.711105:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6903 00000100:00080000:2.0:1788574990.711107:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574990.711109:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574990.711113:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574990.711120:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616468608/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574990.711150:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616468352:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788574990.711241:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574990.711292:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574990/real 1788574990] req@ffff9ada60625880 x1875449616468352/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575015 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574990.711315:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616468352/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575015 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574990.711325:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574990.711329:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574990.711333:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616468352:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574990.711339:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616468480:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574990.711359:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574990.711368:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574990/real 1788574990] req@ffff9ada60625500 x1875449616468480/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575015 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574990.711377:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616468480/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575015 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574990.711383:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574990.711385:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574990.711388:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616468480:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574990.711419:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616468608:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574990.711434:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574990.711446:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574990/real 1788574990] req@ffff9ada60625c00 x1875449616468608/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575015 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574990.711459:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616468608/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575015 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574990.711467:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574990.711471:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574990.711475:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616468608:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574990.903003:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574991.223046:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574991.926946:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574992.247040:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574992.950969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574993.270999:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574993.974911:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574994.295059:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574994.998971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574995.319077:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788574995.830996:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574995.831008:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788574995.831012:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574995.831024:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6908 00000100:00080000:2.0:1788574995.831037:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574995.831045:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788574995.831103:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616468736/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574995.831128:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574995.831132:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788574995.831133:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574995.831136:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6908 00000100:00080000:2.0:1788574995.831139:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574995.831141:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574995.831151:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574995.831160:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616468864/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788574995.831169:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788574995.831173:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788574995.831175:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788574995.831178:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6908 00000100:00080000:2.0:1788574995.831181:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788574995.831184:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788574995.831189:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788574995.831198:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616468992/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788574995.831212:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616468736:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788574995.831288:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574995.831335:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574995/real 1788574995] req@ffff9ada60625f80 x1875449616468736/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575020 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574995.831357:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616468736/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575020 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574995.831366:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574995.831369:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574995.831373:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616468736:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788574995.831379:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616468864:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574995.831398:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574995.831407:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574995/real 1788574995] req@ffff9ada60624e00 x1875449616468864/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575020 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574995.831416:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616468864/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575020 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574995.831422:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574995.831424:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574995.831427:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616468864:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574995.831448:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616468992:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788574995.831460:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788574995.831467:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788574995/real 1788574995] req@ffff9ada60627480 x1875449616468992/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575020 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788574995.831475:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616468992/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575020 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788574995.831480:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788574995.831482:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788574995.831484:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616468992:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788574996.022978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574996.343032:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574997.047001:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574997.367123:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574998.070979:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574998.391033:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788574999.094981:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788574999.415104:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575000.118909:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575000.439114:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788575000.951021:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575000.951029:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788575000.951033:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575000.951042:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6913 00000100:00080000:3.0:1788575000.951052:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575000.951058:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575000.951092:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1180 x1875449616469120/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575000.951137:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575000.951141:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788575000.951144:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575000.951148:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6913 00000100:00080000:3.0:1788575000.951152:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575000.951156:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575000.951166:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575000.951187:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3800 x1875449616469248/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788575000.951194:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616469120:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788575000.951198:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575000.951205:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788575000.951206:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575000.951210:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6913 00000100:00080000:3.0:1788575000.951213:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575000.951215:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575000.951220:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575000.951228:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1880 x1875449616469376/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575000.951263:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575000.951299:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575000/real 1788575000] req@ffff9ada604e1180 x1875449616469120/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575025 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575000.951313:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1180 x1875449616469120/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575025 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575000.951321:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575000.951324:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575000.951327:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616469120:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575000.951350:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616469248:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575000.951374:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575000.951387:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575000/real 1788575000] req@ffff9ada604e3800 x1875449616469248/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575025 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575000.951400:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3800 x1875449616469248/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575025 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575000.951409:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575000.951412:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575000.951417:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616469248:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575000.951423:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616469376:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575000.951441:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575000.951450:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575000/real 1788575000] req@ffff9ada604e1880 x1875449616469376/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575025 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575000.951458:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1880 x1875449616469376/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575025 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575000.951463:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575000.951465:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575000.951467:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616469376:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575001.142973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575001.463111:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575002.166980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575002.487108:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575003.191024:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575003.511070:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575004.214965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575004.534994:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575005.238962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575005.558978:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788575006.070976:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575006.070996:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788575006.071001:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575006.071011:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6918 00000100:00080000:3.0:1788575006.071022:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575006.071029:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575006.071066:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2d80 x1875449616469504/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575006.071095:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575006.071098:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788575006.071100:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575006.071104:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6918 00000100:00080000:3.0:1788575006.071107:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575006.071111:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575006.071120:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575006.071133:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0a80 x1875449616469632/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575006.071144:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575006.071148:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788575006.071150:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575006.071157:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6918 00000100:00100000:0.0:1788575006.071159:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616469504:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788575006.071161:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575006.071170:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575006.071176:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575006.071191:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2300 x1875449616469760/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575006.071238:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575006.071273:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575006/real 1788575006] req@ffff9ada604e2d80 x1875449616469504/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575031 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575006.071293:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2d80 x1875449616469504/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575031 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575006.071305:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575006.071309:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575006.071314:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616469504:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575006.071322:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616469632:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575006.071367:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575006.071381:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575006/real 1788575006] req@ffff9ada604e0a80 x1875449616469632/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575031 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575006.071395:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0a80 x1875449616469632/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575031 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575006.071405:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575006.071409:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575006.071412:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616469632:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575006.071434:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616469760:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575006.071450:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575006.071458:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575006/real 1788575006] req@ffff9ada604e2300 x1875449616469760/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575031 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575006.071467:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2300 x1875449616469760/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575031 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575006.071472:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575006.071475:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575006.071477:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616469760:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575006.262976:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575006.583042:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575007.286966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575007.607056:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575008.310961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575008.630994:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575009.334953:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575009.654979:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575010.358948:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575010.679009:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575011.190970:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575011.190977:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575011.190981:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575011.190990:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6923 00000100:00080000:2.0:1788575011.190999:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575011.191004:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575011.191038:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616469888/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575011.191059:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575011.191063:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575011.191065:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575011.191069:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6923 00000100:00080000:2.0:1788575011.191072:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575011.191076:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575011.191087:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575011.191100:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616470016/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575011.191110:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575011.191113:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:0.0:1788575011.191114:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616469888:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788575011.191115:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575011.191118:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6923 00000100:00080000:2.0:1788575011.191123:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575011.191126:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575011.191131:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575011.191140:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616470144/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575011.191175:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575011.191205:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575011/real 1788575011] req@ffff9ada60625180 x1875449616469888/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575036 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575011.191218:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616469888/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575036 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575011.191225:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575011.191228:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575011.191231:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616469888:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575011.191251:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616470016:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575011.191277:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575011.191291:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575011/real 1788575011] req@ffff9ada60626d80 x1875449616470016/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575036 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575011.191316:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616470016/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575036 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575011.191323:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575011.191326:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575011.191329:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616470016:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575011.191336:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616470144:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575011.191359:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575011.191372:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575011/real 1788575011] req@ffff9ada60627b80 x1875449616470144/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575036 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575011.191391:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616470144/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575036 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575011.191401:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575011.191404:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575011.191408:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616470144:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575011.382974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575011.703052:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575012.406987:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575012.727019:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575013.430921:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575013.751054:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575014.454981:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575014.775255:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575015.478995:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575015.799043:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575016.310996:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575016.311004:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575016.311008:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575016.311017:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6928 00000100:00080000:1.0:1788575016.311026:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575016.311032:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575016.311064:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f800 x1875449616470272/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575016.311087:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575016.311091:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575016.311093:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575016.311097:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6928 00000100:00080000:1.0:1788575016.311100:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575016.311104:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575016.311114:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575016.311129:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566dc00 x1875449616470400/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575016.311142:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1788575016.311144:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616470272:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788575016.311148:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788575016.311153:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575016.311159:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6928 00000100:00080000:1.0:1788575016.311163:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575016.311168:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575016.311173:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575016.311187:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ed80 x1875449616470528/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575016.311221:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575016.311259:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575016/real 1788575016] req@ffff9ada4566f800 x1875449616470272/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575041 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575016.311279:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f800 x1875449616470272/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575041 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575016.311291:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575016.311296:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575016.311300:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616470272:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575016.311325:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616470400:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575016.311363:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575016.311384:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575016/real 1788575016] req@ffff9ada4566dc00 x1875449616470400/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575041 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575016.311406:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566dc00 x1875449616470400/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575041 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575016.311415:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575016.311419:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575016.311422:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616470400:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575016.311438:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616470528:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575016.311462:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575016.311474:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575016/real 1788575016] req@ffff9ada4566ed80 x1875449616470528/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575041 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575016.311488:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ed80 x1875449616470528/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575041 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575016.311496:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575016.311500:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575016.311504:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616470528:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575016.503017:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575016.823031:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575017.526973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575017.846991:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575018.550966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575018.871063:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575019.574982:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575019.895066:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575020.598976:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575020.919045:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575021.430941:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575021.430954:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575021.430959:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575021.430970:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6933 00000100:00080000:1.0:1788575021.430982:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575021.430989:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575021.431039:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c000 x1875449616470656/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575021.431074:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575021.431077:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575021.431079:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575021.431082:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6933 00000100:00080000:1.0:1788575021.431086:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575021.431089:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575021.431099:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575021.431110:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566df80 x1875449616470784/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575021.431120:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1788575021.431123:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616470656:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788575021.431124:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788575021.431126:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575021.431133:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6933 00000100:00080000:1.0:1788575021.431138:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575021.431140:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575021.431143:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575021.431150:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566fb80 x1875449616470912/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575021.431219:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575021.431268:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575021/real 1788575021] req@ffff9ada4566c000 x1875449616470656/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575046 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575021.431292:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c000 x1875449616470656/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575046 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575021.431313:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575021.431319:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575021.431324:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616470656:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575021.431355:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616470784:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575021.431383:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575021.431398:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575021/real 1788575021] req@ffff9ada4566df80 x1875449616470784/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575046 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575021.431412:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566df80 x1875449616470784/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575046 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575021.431421:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575021.431425:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575021.431429:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616470784:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575021.431447:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616470912:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575021.431472:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575021.431484:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575021/real 1788575021] req@ffff9ada4566fb80 x1875449616470912/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575046 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575021.431498:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566fb80 x1875449616470912/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575046 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575021.431507:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575021.431511:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575021.431515:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616470912:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575021.622970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575021.943152:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575022.646965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575022.967044:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575023.670985:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575023.991066:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575024.695000:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575025.015068:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575025.718988:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575026.039060:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575026.550965:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575026.550973:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575026.550977:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575026.550985:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6939 00000100:00080000:1.0:1788575026.550994:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575026.551000:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575026.551032:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566fb80 x1875449616471040/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575026.551055:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575026.551058:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575026.551059:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575026.551062:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6939 00000100:00080000:1.0:1788575026.551064:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575026.551066:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575026.551073:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575026.551083:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566df80 x1875449616471168/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575026.551096:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575026.551099:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788575026.551102:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1788575026.551104:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616471040:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788575026.551107:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6939 00000100:00080000:1.0:1788575026.551112:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575026.551115:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575026.551118:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575026.551128:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c000 x1875449616471296/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575026.551166:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575026.551196:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575026/real 1788575026] req@ffff9ada4566fb80 x1875449616471040/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575051 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575026.551210:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566fb80 x1875449616471040/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575051 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575026.551218:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575026.551220:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575026.551223:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616471040:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575026.551240:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616471168:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575026.551255:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575026.551263:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575026/real 1788575026] req@ffff9ada4566df80 x1875449616471168/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575051 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575026.551271:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566df80 x1875449616471168/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575051 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575026.551277:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575026.551279:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575026.551282:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616471168:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575026.551285:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616471296:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575026.551296:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575026.551301:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575026/real 1788575026] req@ffff9ada4566c000 x1875449616471296/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575051 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575026.551310:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c000 x1875449616471296/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575051 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575026.551315:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575026.551316:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575026.551318:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616471296:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575026.742939:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575027.063040:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575027.766979:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575028.086996:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575028.790978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575029.111056:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575029.814971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575030.135077:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575030.838959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575031.159032:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575031.670999:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575031.671005:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575031.671009:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575031.671017:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6944 00000100:00080000:0.0:1788575031.671025:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575031.671031:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575031.671061:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616471424/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575031.671086:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575031.671089:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575031.671091:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575031.671095:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6944 00000100:00080000:0.0:1788575031.671098:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575031.671101:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575031.671110:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575031.671121:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616471552/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575031.671135:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575031.671138:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575031.671140:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575031.671143:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6944 00000100:00100000:3.0:1788575031.671144:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616471424:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788575031.671146:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575031.671149:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575031.671153:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575031.671162:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616471680/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575031.671208:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575031.671235:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575031/real 1788575031] req@ffff9ada4557c700 x1875449616471424/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575056 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575031.671247:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616471424/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575056 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575031.671254:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575031.671257:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575031.671259:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616471424:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575031.671274:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616471552:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575031.671288:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575031.671295:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575031/real 1788575031] req@ffff9ada4557f480 x1875449616471552/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575056 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575031.671301:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616471552/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575056 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575031.671306:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575031.671308:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575031.671310:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616471552:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575031.671313:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616471680:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575031.671323:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575031.671328:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575031/real 1788575031] req@ffff9ada4557ca80 x1875449616471680/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575056 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575031.671339:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616471680/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575056 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575031.671346:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575031.671349:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575031.671351:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616471680:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575031.862964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575032.183002:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575032.886961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575033.207032:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575033.910984:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575034.231010:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575034.934955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575035.254990:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575035.959023:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575036.279010:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575036.790993:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575036.791000:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575036.791004:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575036.791011:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6949 00000100:00080000:2.0:1788575036.791020:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575036.791026:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575036.791057:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626300 x1875449616471808/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575036.791098:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575036.791102:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575036.791104:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575036.791107:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6949 00000100:00080000:2.0:1788575036.791111:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575036.791114:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575036.791123:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575036.791135:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627800 x1875449616471936/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575036.791144:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575036.791148:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575036.791150:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575036.791153:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6949 00000100:00080000:2.0:1788575036.791156:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575036.791160:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575036.791165:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575036.791181:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616472064/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575036.791214:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616471808:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788575036.791264:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575036.791290:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575036/real 1788575036] req@ffff9ada60626300 x1875449616471808/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575061 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575036.791307:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626300 x1875449616471808/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575061 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575036.791313:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575036.791316:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575036.791318:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616471808:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575036.791323:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616471936:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575036.791335:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575036.791342:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575036/real 1788575036] req@ffff9ada60627800 x1875449616471936/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575061 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575036.791349:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627800 x1875449616471936/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575061 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575036.791353:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575036.791355:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575036.791356:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616471936:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575036.791359:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616472064:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575036.791368:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575036.791373:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575036/real 1788575036] req@ffff9ada60627b80 x1875449616472064/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575061 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575036.791380:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616472064/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575061 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575036.791388:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575036.791389:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575036.791391:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616472064:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575036.983041:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575037.302979:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575038.006954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575038.327066:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575039.030970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575039.351021:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575040.054940:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575040.375055:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575041.078952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575041.399028:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575041.910967:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575041.910974:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575041.910977:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575041.910984:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6954 00000100:00080000:2.0:1788575041.910996:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575041.911002:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575041.911036:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616472192/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575041.911062:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575041.911065:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575041.911066:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575041.911069:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6954 00000100:00080000:2.0:1788575041.911072:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575041.911074:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575041.911081:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575041.911090:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616472320/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575041.911098:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575041.911101:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575041.911102:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575041.911105:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6954 00000100:00080000:2.0:1788575041.911107:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575041.911110:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575041.911113:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575041.911118:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616472448/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575041.911145:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616472192:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788575041.911244:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575041.911283:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575041/real 1788575041] req@ffff9ada60626d80 x1875449616472192/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575066 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575041.911300:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616472192/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575066 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575041.911319:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575041.911324:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575041.911328:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616472192:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575041.911333:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616472320:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575041.911350:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575041.911359:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575041/real 1788575041] req@ffff9ada60625180 x1875449616472320/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575066 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575041.911367:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616472320/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575066 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575041.911372:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575041.911374:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575041.911376:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616472320:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575041.911380:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616472448:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575041.911390:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575041.911395:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575041/real 1788575041] req@ffff9ada60627480 x1875449616472448/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575066 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575041.911402:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616472448/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575066 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575041.911406:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575041.911408:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575041.911410:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616472448:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575042.102956:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575042.423040:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575043.126976:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575043.447066:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575044.150986:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575044.471084:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575045.174945:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575045.495098:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575046.198995:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575046.519020:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575047.031001:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575047.031008:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575047.031012:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575047.031020:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6959 00000100:00080000:0.0:1788575047.031027:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575047.031032:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575047.031059:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616472576/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575047.031079:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575047.031081:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575047.031083:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575047.031085:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6959 00000100:00080000:0.0:1788575047.031087:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575047.031089:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575047.031096:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575047.031102:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616472704/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575047.031108:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575047.031111:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575047.031112:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575047.031114:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6959 00000100:00080000:0.0:1788575047.031116:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575047.031118:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575047.031120:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575047.031126:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616472832/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575047.031131:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616472576:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575047.031201:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616472704:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575047.031226:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616472832:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575047.222958:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575047.542975:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575048.246953:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575048.567005:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575049.270964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575049.590980:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1788575050.103162:0:2372:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:1.0:1788575050.103205:0:2372:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1788575050.103219:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:1.0:1788575050.103260:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:1.0:1788575050.103269:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:3.0:1788575050.103320:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575047/real 1788575050] req@ffff9ada4557d880 x1875449616472832/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575072 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575050.103347:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616472832/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575072 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575050.103359:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575050.103364:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575050.103396:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616472832:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575050.103403:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575047/real 1788575050] req@ffff9ada4557dc00 x1875449616472576/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575072 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575050.103416:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616472576/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575072 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575050.103424:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575050.103428:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575050.103430:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616472576:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575050.103436:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575047/real 1788575050] req@ffff9ada4557f800 x1875449616472704/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575072 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575050.103447:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616472704/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575072 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575050.103454:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575050.103457:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575050.103461:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616472704:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575050.294951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575050.615030:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575051.318947:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575051.639028:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575052.150954:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575052.150961:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575052.150964:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575052.150972:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6964 00000100:00080000:0.0:1788575052.150980:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575052.150985:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575052.151013:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616472960/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575052.151036:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575052.151038:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575052.151039:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575052.151042:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6964 00000100:00080000:0.0:1788575052.151044:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575052.151046:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575052.151053:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575052.151060:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616473088/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575052.151066:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575052.151069:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575052.151070:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575052.151074:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6964 00000100:00080000:0.0:1788575052.151077:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575052.151086:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575052.151095:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575052.151097:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616472960:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575052.151108:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616473216/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575052.151157:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575052.151188:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575052/real 1788575052] req@ffff9ada4557e300 x1875449616472960/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575077 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575052.151206:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616472960/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575077 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575052.151216:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575052.151220:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575052.151224:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616472960:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575052.151232:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616473088:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575052.151254:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575052.151265:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575052/real 1788575052] req@ffff9ada4557fb80 x1875449616473088/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575077 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575052.151277:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616473088/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575077 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575052.151285:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575052.151288:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575052.151290:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616473088:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575052.151312:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616473216:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575052.151332:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575052.151342:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575052/real 1788575052] req@ffff9ada4557ed80 x1875449616473216/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575077 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575052.151352:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616473216/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575077 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575052.151359:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575052.151361:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575052.151364:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616473216:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575052.342947:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575052.663004:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575053.366959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575053.687027:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575054.390930:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575054.711022:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575055.414983:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575055.735066:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575056.438961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575056.758973:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575057.270980:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575057.270989:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575057.270992:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575057.270999:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6969 00000100:00080000:0.0:1788575057.271006:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575057.271012:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575057.271043:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616473344/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575057.271070:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575057.271075:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575057.271076:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575057.271079:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6969 00000100:00080000:0.0:1788575057.271081:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575057.271084:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575057.271092:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575057.271102:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616473472/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575057.271109:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616473344:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788575057.271111:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575057.271116:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575057.271117:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575057.271120:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6969 00000100:00080000:0.0:1788575057.271122:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575057.271124:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575057.271127:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575057.271134:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557df80 x1875449616473600/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575057.271163:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575057.271189:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575057/real 1788575057] req@ffff9ada4557ea00 x1875449616473344/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575082 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575057.271212:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616473344/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575082 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575057.271222:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575057.271226:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575057.271229:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616473344:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575057.271250:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616473472:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575057.271266:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575057.271274:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575057/real 1788575057] req@ffff9ada4557e680 x1875449616473472/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575082 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575057.271282:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616473472/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575082 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575057.271287:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575057.271289:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575057.271291:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616473472:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575057.271294:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616473600:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575057.271305:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575057.271311:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575057/real 1788575057] req@ffff9ada4557df80 x1875449616473600/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575082 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575057.271318:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557df80 x1875449616473600/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575082 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575057.271322:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575057.271323:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575057.271325:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616473600:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575057.462929:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575057.783010:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575058.486961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575058.806948:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575059.510977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575059.831033:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575060.534970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575060.855053:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575061.558957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575061.879081:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575062.390946:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575062.390957:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575062.390963:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575062.390974:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6974 00000100:00080000:0.0:1788575062.390985:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575062.390995:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575062.391053:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d500 x1875449616473728/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575062.391079:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575062.391083:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575062.391085:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575062.391090:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6974 00000100:00080000:0.0:1788575062.391094:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575062.391100:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575062.391113:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575062.391130:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d180 x1875449616473856/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575062.391130:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616473728:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788575062.391149:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575062.391152:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575062.391153:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575062.391157:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6974 00000100:00080000:0.0:1788575062.391161:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575062.391163:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575062.391168:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575062.391179:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557df80 x1875449616473984/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575062.391221:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575062.391269:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575062/real 1788575062] req@ffff9ada4557d500 x1875449616473728/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575087 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575062.391294:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d500 x1875449616473728/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575087 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575062.391308:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575062.391313:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575062.391318:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616473728:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575062.391344:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616473856:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575062.391367:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575062.391380:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575062/real 1788575062] req@ffff9ada4557d180 x1875449616473856/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575087 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575062.391393:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d180 x1875449616473856/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575087 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575062.391401:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575062.391405:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575062.391409:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616473856:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575062.391415:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616473984:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575062.391433:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575062.391442:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575062/real 1788575062] req@ffff9ada4557df80 x1875449616473984/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575087 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575062.391454:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557df80 x1875449616473984/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575087 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575062.391462:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575062.391466:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575062.391470:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616473984:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575062.582975:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575062.903044:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575063.607010:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575063.927113:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575064.631009:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575064.951073:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575065.655016:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575065.975104:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575066.678969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575066.998992:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575067.511000:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575067.511009:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575067.511013:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575067.511021:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6980 00000100:00080000:2.0:1788575067.511030:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575067.511036:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575067.511069:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616474112/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575067.511093:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575067.511096:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575067.511097:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575067.511100:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6980 00000100:00080000:2.0:1788575067.511102:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575067.511104:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575067.511113:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575067.511121:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616474240/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575067.511129:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575067.511131:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575067.511133:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575067.511135:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6980 00000100:00080000:2.0:1788575067.511137:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575067.511144:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575067.511146:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616474112:192.168.202.113@tcp:250:kworker.0 00000008:00100000:2.0:1788575067.511147:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575067.511159:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616474368/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575067.511214:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575067.511262:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575067/real 1788575067] req@ffff9ada60624e00 x1875449616474112/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575092 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575067.511280:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616474112/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575092 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575067.511290:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575067.511293:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575067.511297:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616474112:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575067.511304:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616474240:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575067.511326:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575067.511339:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575067/real 1788575067] req@ffff9ada60625f80 x1875449616474240/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575092 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575067.511348:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616474240/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575092 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575067.511353:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575067.511356:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575067.511358:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616474240:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575067.511377:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616474368:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575067.511396:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575067.511410:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575067/real 1788575067] req@ffff9ada60625c00 x1875449616474368/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575092 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575067.511423:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616474368/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575092 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575067.511432:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575067.511436:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575067.511439:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616474368:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575067.702959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575068.023033:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575068.726973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575069.047052:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575069.750967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575070.071065:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575070.775005:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575071.095097:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575071.798930:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575072.119210:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575072.631005:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575072.631013:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575072.631017:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575072.631026:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6985 00000100:00080000:1.0:1788575072.631035:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575072.631042:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575072.631074:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ed80 x1875449616474496/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575072.631101:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575072.631105:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575072.631107:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575072.631111:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6985 00000100:00080000:1.0:1788575072.631115:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575072.631119:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575072.631129:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575072.631145:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566dc00 x1875449616474624/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575072.631147:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616474496:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788575072.631162:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575072.631164:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788575072.631166:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575072.631169:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6985 00000100:00080000:1.0:1788575072.631172:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575072.631174:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575072.631179:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575072.631188:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f800 x1875449616474752/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575072.631207:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575072.631237:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575072/real 1788575072] req@ffff9ada4566ed80 x1875449616474496/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575097 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575072.631251:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ed80 x1875449616474496/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575097 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575072.631259:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575072.631262:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575072.631265:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616474496:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575072.631284:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616474624:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575072.631298:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575072.631305:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575072/real 1788575072] req@ffff9ada4566dc00 x1875449616474624/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575097 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575072.631313:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566dc00 x1875449616474624/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575097 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575072.631319:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575072.631321:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575072.631323:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616474624:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575072.631326:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616474752:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575072.631336:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575072.631346:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575072/real 1788575072] req@ffff9ada4566f800 x1875449616474752/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575097 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575072.631358:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f800 x1875449616474752/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575097 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575072.631367:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575072.631371:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575072.631374:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616474752:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575072.822971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575073.143044:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575073.847013:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575074.167025:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575074.871004:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575075.191007:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575075.895113:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575076.214966:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575076.919004:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575077.238995:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575077.750940:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575077.750950:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575077.750955:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575077.750976:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6990 00000100:00080000:1.0:1788575077.750985:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575077.750991:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575077.751024:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e300 x1875449616474880/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575077.751049:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575077.751053:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575077.751055:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575077.751059:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6990 00000100:00080000:1.0:1788575077.751062:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575077.751067:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575077.751076:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575077.751083:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616474880:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788575077.751090:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f480 x1875449616475008/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575077.751102:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575077.751105:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788575077.751107:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575077.751110:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6990 00000100:00080000:1.0:1788575077.751119:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575077.751123:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575077.751128:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1788575077.751139:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:1.0:1788575077.751140:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ea00 x1875449616475136/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575077.751182:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575077/real 1788575077] req@ffff9ada4566e300 x1875449616474880/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575102 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575077.751200:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e300 x1875449616474880/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575102 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575077.751210:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575077.751214:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575077.751218:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616474880:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575077.751238:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616475008:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575077.751257:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575077.751267:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575077/real 1788575077] req@ffff9ada4566f480 x1875449616475008/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575102 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575077.751278:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f480 x1875449616475008/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575102 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575077.751284:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575077.751286:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575077.751288:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616475008:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575077.751292:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616475136:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575077.751304:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575077.751310:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575077/real 1788575077] req@ffff9ada4566ea00 x1875449616475136/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575102 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575077.751317:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ea00 x1875449616475136/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575102 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575077.751321:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575077.751323:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575077.751325:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616475136:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575077.942959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575078.262999:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575078.966929:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575079.287060:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575079.990954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575080.311043:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575081.014912:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575081.335056:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575082.038976:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575082.359056:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575082.870956:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575082.870963:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575082.870966:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575082.870973:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 6995 00000100:00080000:1.0:1788575082.870982:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575082.870987:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575082.871019:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d500 x1875449616475264/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575082.871042:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575082.871045:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575082.871046:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575082.871071:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6995 00000100:00080000:1.0:1788575082.871074:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575082.871077:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575082.871084:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575082.871098:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ca80 x1875449616475392/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575082.871106:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575082.871109:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1788575082.871109:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616475264:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788575082.871112:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575082.871119:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 6995 00000100:00080000:1.0:1788575082.871123:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575082.871126:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575082.871131:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575082.871143:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d180 x1875449616475520/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575082.871177:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575082.871209:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575082/real 1788575082] req@ffff9ada4566d500 x1875449616475264/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575107 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575082.871227:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d500 x1875449616475264/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575107 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575082.871236:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575082.871241:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575082.871244:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616475264:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575082.871266:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616475392:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575082.871287:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575082.871298:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575082/real 1788575082] req@ffff9ada4566ca80 x1875449616475392/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575107 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575082.871309:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ca80 x1875449616475392/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575107 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575082.871317:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575082.871320:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575082.871323:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616475392:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575082.871328:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616475520:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575082.871345:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575082.871354:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575082/real 1788575082] req@ffff9ada4566d180 x1875449616475520/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575107 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575082.871364:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d180 x1875449616475520/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575107 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575082.871371:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575082.871374:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575082.871377:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616475520:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575083.062962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575083.383029:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575084.086957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575084.407006:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575085.110960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575085.431029:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575086.134905:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575086.455025:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575087.158994:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575087.478993:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575087.990974:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575087.990981:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575087.990984:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575087.990991:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7000 00000100:00080000:2.0:1788575087.990999:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575087.991005:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575087.991033:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616475648/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575087.991059:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575087.991062:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575087.991063:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575087.991065:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7000 00000100:00080000:2.0:1788575087.991068:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575087.991071:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575087.991077:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575087.991085:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616475776/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575087.991091:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575087.991093:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575087.991094:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575087.991100:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7000 00000100:00080000:2.0:1788575087.991104:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575087.991106:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575087.991109:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575087.991115:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616475904/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575087.991157:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616475648:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788575087.991243:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575087.991292:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575087/real 1788575087] req@ffff9ada60625500 x1875449616475648/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575112 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575087.991313:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616475648/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575112 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575087.991326:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575087.991331:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575087.991335:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616475648:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575087.991344:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616475776:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575087.991369:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575087.991382:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575087/real 1788575087] req@ffff9ada60625880 x1875449616475776/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575112 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575087.991395:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616475776/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575112 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575087.991403:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575087.991406:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575087.991410:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616475776:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575087.991444:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616475904:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575087.991466:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575087.991478:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575087/real 1788575087] req@ffff9ada60624a80 x1875449616475904/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575112 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575087.991490:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616475904/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575112 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575087.991499:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575087.991502:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575087.991506:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616475904:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575088.182978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575088.503013:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575089.206955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575089.527043:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575090.230952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575090.551010:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575091.254959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575091.575016:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575092.278968:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575092.599031:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788575093.110995:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575093.111002:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788575093.111007:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575093.111017:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7005 00000100:00080000:3.0:1788575093.111026:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575093.111034:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575093.111078:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0000 x1875449616476032/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575093.111103:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575093.111106:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788575093.111107:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575093.111110:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7005 00000100:00080000:3.0:1788575093.111114:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575093.111116:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575093.111123:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575093.111132:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1c00 x1875449616476160/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575093.111139:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575093.111143:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788575093.111144:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575093.111148:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7005 00000100:00080000:3.0:1788575093.111152:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575093.111155:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575093.111159:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575093.111163:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616476032:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575093.111172:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3b80 x1875449616476288/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575093.111233:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575093.111266:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575093/real 1788575093] req@ffff9ada604e0000 x1875449616476032/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575118 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575093.111282:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0000 x1875449616476032/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575118 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575093.111292:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575093.111296:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575093.111300:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616476032:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575093.111306:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616476160:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575093.111326:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575093.111336:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575093/real 1788575093] req@ffff9ada604e1c00 x1875449616476160/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575118 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575093.111348:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1c00 x1875449616476160/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575118 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575093.111356:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575093.111359:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575093.111362:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616476160:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575093.111383:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616476288:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575093.111402:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575093.111412:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575093/real 1788575093] req@ffff9ada604e3b80 x1875449616476288/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575118 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575093.111423:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3b80 x1875449616476288/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575118 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575093.111431:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575093.111434:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575093.111437:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616476288:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575093.302998:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575093.623006:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575094.326977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575094.647053:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575095.350992:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575095.670984:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575096.374988:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575096.695092:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575097.398999:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575097.718979:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788575098.230996:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575098.231006:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788575098.231010:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575098.231020:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7010 00000100:00080000:3.0:1788575098.231031:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575098.231038:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575098.231081:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1f80 x1875449616476416/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575098.231108:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575098.231111:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788575098.231112:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575098.231115:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7010 00000100:00080000:3.0:1788575098.231117:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575098.231119:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575098.231129:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575098.231138:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2680 x1875449616476544/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575098.231144:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575098.231147:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788575098.231148:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575098.231150:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7010 00000100:00080000:3.0:1788575098.231152:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575098.231154:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575098.231156:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575098.231162:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616476416:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575098.231164:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3100 x1875449616476672/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575098.231229:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575098.231262:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575098/real 1788575098] req@ffff9ada604e1f80 x1875449616476416/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575123 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575098.231276:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1f80 x1875449616476416/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575123 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575098.231288:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575098.231291:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575098.231294:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616476416:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575098.231299:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616476544:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575098.231327:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575098.231335:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575098/real 1788575098] req@ffff9ada604e2680 x1875449616476544/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575123 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575098.231343:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2680 x1875449616476544/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575123 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575098.231351:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575098.231354:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575098.231357:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616476544:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575098.231385:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616476672:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575098.231401:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575098.231408:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575098/real 1788575098] req@ffff9ada604e3100 x1875449616476672/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575123 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575098.231415:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3100 x1875449616476672/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575123 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575098.231421:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575098.231423:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575098.231425:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616476672:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575098.422983:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575098.743076:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575099.446970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575099.767042:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575100.470962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575100.791063:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575101.494975:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575101.815038:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575102.518972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575102.839037:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788575103.350973:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575103.350983:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788575103.350987:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575103.350997:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7015 00000100:00080000:3.0:1788575103.351007:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575103.351015:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575103.351053:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2a00 x1875449616476800/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575103.351076:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575103.351078:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788575103.351080:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575103.351083:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7015 00000100:00080000:3.0:1788575103.351084:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575103.351086:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575103.351095:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575103.351106:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1500 x1875449616476928/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575103.351113:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575103.351116:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788575103.351119:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1788575103.351121:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616476800:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788575103.351123:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7015 00000100:00080000:3.0:1788575103.351128:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575103.351131:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575103.351134:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575103.351144:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0e00 x1875449616477056/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575103.351206:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575103.351248:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575103/real 1788575103] req@ffff9ada604e2a00 x1875449616476800/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575128 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575103.351273:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2a00 x1875449616476800/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575128 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575103.351284:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575103.351288:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575103.351292:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616476800:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575103.351316:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616476928:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575103.351334:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575103.351342:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575103/real 1788575103] req@ffff9ada604e1500 x1875449616476928/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575128 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575103.351350:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1500 x1875449616476928/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575128 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575103.351355:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575103.351357:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575103.351360:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616476928:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575103.351363:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616477056:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575103.351373:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575103.351378:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575103/real 1788575103] req@ffff9ada604e0e00 x1875449616477056/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575128 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575103.351385:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0e00 x1875449616477056/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575128 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575103.351403:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575103.351406:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575103.351407:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616477056:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575103.542981:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575103.863071:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575104.566956:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575104.887044:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575105.590944:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575105.911019:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575106.614974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575106.935050:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575107.638975:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575107.958997:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575108.470974:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575108.470984:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575108.470988:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575108.470997:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7021 00000100:00080000:0.0:1788575108.471007:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575108.471014:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575108.471056:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616477184/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575108.471087:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575108.471091:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575108.471093:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575108.471097:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7021 00000100:00080000:0.0:1788575108.471101:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575108.471106:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575108.471117:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575108.471120:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616477184:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575108.471133:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616477312/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575108.471146:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575108.471149:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575108.471151:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575108.471154:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7021 00000100:00080000:0.0:1788575108.471159:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575108.471163:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575108.471167:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575108.471179:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616477440/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575108.471207:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575108.471257:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575108/real 1788575108] req@ffff9ada4557e680 x1875449616477184/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575133 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575108.471277:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616477184/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575133 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575108.471288:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575108.471292:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575108.471294:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616477184:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575108.471318:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616477312:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575108.471340:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575108.471352:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575108/real 1788575108] req@ffff9ada4557ea00 x1875449616477312/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575133 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575108.471363:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616477312/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575133 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575108.471369:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575108.471371:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575108.471373:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616477312:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575108.471377:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616477440:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575108.471391:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575108.471397:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575108/real 1788575108] req@ffff9ada4557ed80 x1875449616477440/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575133 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575108.471408:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616477440/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575133 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575108.471413:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575108.471414:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575108.471416:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616477440:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575108.662989:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575108.983042:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575109.686974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575110.007074:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575110.710939:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575111.031042:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575111.734970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575112.055059:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575112.758976:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575113.079049:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788575113.590952:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575113.590962:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788575113.590967:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575113.590978:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7026 00000100:00080000:3.0:1788575113.590989:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575113.590996:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575113.591039:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0e00 x1875449616477568/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575113.591078:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575113.591082:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788575113.591084:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575113.591089:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7026 00000100:00080000:3.0:1788575113.591093:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575113.591098:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575113.591110:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575113.591112:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616477568:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575113.591123:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1500 x1875449616477696/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575113.591132:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575113.591135:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788575113.591136:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575113.591139:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7026 00000100:00080000:3.0:1788575113.591142:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575113.591144:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575113.591147:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575113.591155:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2a00 x1875449616477824/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788575113.591222:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616477696:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575113.591252:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616477824:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575113.782965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575114.103051:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575114.806985:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575115.126988:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575115.830957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575116.151011:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1788575116.663171:0:2373:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:2.0:1788575116.663236:0:2373:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1788575116.663256:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788575116.663301:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788575116.663315:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:0.0:1788575116.663369:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575113/real 1788575116] req@ffff9ada604e1500 x1875449616477696/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575138 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575116.663401:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1500 x1875449616477696/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575138 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575116.663417:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575116.663422:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575116.663428:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616477696:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575116.663437:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575113/real 1788575116] req@ffff9ada604e2a00 x1875449616477824/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575138 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575116.663450:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2a00 x1875449616477824/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575138 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575116.663458:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575116.663462:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575116.663466:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616477824:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575116.663475:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575113/real 1788575116] req@ffff9ada604e0e00 x1875449616477568/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575138 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575116.663487:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0e00 x1875449616477568/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575138 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575116.663495:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575116.663499:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575116.663503:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616477568:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575116.854977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575117.175049:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575117.879002:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575118.199052:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575118.710993:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575118.711004:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575118.711008:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575118.711018:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7031 00000100:00080000:1.0:1788575118.711030:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575118.711040:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575118.711090:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f100 x1875449616477952/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575118.711126:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575118.711130:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575118.711133:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575118.711138:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7031 00000100:00080000:1.0:1788575118.711151:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575118.711155:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575118.711168:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575118.711187:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c380 x1875449616478080/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575118.711196:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575118.711199:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788575118.711201:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575118.711205:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7031 00000100:00080000:1.0:1788575118.711208:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575118.711211:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575118.711216:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575118.711226:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c700 x1875449616478208/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788575118.711329:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616477952:192.168.202.113@tcp:250:kworker.0 00000800:00000100:0.0:1788575118.711431:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575118.711476:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575118/real 1788575118] req@ffff9ada4566f100 x1875449616477952/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575143 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575118.711497:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f100 x1875449616477952/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575143 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575118.711506:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575118.711509:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575118.711513:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616477952:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575118.711521:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616478080:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575118.711563:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575118.711576:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575118/real 1788575118] req@ffff9ada4566c380 x1875449616478080/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575143 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575118.711595:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c380 x1875449616478080/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575143 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575118.711602:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575118.711605:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575118.711609:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616478080:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575118.711614:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616478208:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575118.711633:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575118.711642:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575118/real 1788575118] req@ffff9ada4566c700 x1875449616478208/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575143 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575118.711680:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c700 x1875449616478208/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575143 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575118.711687:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575118.711690:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575118.711693:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616478208:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575118.902981:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575119.223037:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575119.926941:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575120.247055:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575120.950957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575121.271062:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575121.974984:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575122.295076:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575122.998976:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575123.319062:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575123.830987:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575123.830996:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575123.831000:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575123.831009:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7036 00000100:00080000:0.0:1788575123.831020:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575123.831028:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575123.831062:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616478336/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575123.831090:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575123.831094:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575123.831095:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575123.831099:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7036 00000100:00080000:0.0:1788575123.831102:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575123.831106:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575123.831114:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575123.831126:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616478464/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575123.831136:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575123.831140:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1788575123.831140:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616478336:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788575123.831143:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575123.831151:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7036 00000100:00080000:0.0:1788575123.831155:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575123.831158:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575123.831163:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575123.831175:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616478592/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575123.831209:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575123.831239:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575123/real 1788575123] req@ffff9ada4557fb80 x1875449616478336/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575148 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575123.831257:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616478336/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575148 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575123.831267:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575123.831271:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575123.831275:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616478336:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575123.831296:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616478464:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575123.831317:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575123.831339:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575123/real 1788575123] req@ffff9ada4557e300 x1875449616478464/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575148 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575123.831354:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616478464/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575148 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575123.831362:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575123.831365:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575123.831368:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616478464:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575123.831374:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616478592:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575123.831391:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575123.831401:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575123/real 1788575123] req@ffff9ada4557f800 x1875449616478592/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575148 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575123.831412:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616478592/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575148 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575123.831419:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575123.831422:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575123.831425:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616478592:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575124.022980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575124.343076:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575125.046971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575125.367044:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575126.070971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575126.391003:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575127.095017:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575127.415047:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575128.118922:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575128.439019:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575128.951021:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575128.951034:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575128.951039:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575128.951051:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7041 00000100:00080000:1.0:1788575128.951064:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575128.951073:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575128.951113:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ce00 x1875449616478720/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575128.951140:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575128.951144:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575128.951146:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575128.951150:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7041 00000100:00080000:1.0:1788575128.951153:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575128.951157:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575128.951169:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575128.951185:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d880 x1875449616478848/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575128.951198:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616478720:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788575128.951203:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575128.951208:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788575128.951209:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575128.951214:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7041 00000100:00080000:1.0:1788575128.951218:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575128.951223:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575128.951227:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575128.951237:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e680 x1875449616478976/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575128.951279:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575128.951320:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575128/real 1788575128] req@ffff9ada4566ce00 x1875449616478720/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575153 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575128.951338:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ce00 x1875449616478720/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575153 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575128.951349:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575128.951353:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575128.951358:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616478720:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575128.951384:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616478848:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575128.951406:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575128.951418:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575128/real 1788575128] req@ffff9ada4566d880 x1875449616478848/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575153 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575128.951429:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d880 x1875449616478848/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575153 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575128.951438:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575128.951441:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575128.951444:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616478848:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575128.951450:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616478976:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575128.951472:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575128.951483:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575128/real 1788575128] req@ffff9ada4566e680 x1875449616478976/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575153 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575128.951495:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e680 x1875449616478976/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575153 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575128.951503:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575128.951507:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575128.951510:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616478976:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575129.142980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575129.463036:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575130.166950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575130.487033:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575131.190998:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575131.511076:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575132.214922:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575132.535011:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575133.239002:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575133.559057:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575134.070972:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575134.070985:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575134.070991:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575134.071003:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7046 00000100:00080000:1.0:1788575134.071016:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575134.071024:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575134.071074:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e680 x1875449616479104/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575134.071103:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575134.071111:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575134.071113:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575134.071117:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7046 00000100:00080000:1.0:1788575134.071121:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575134.071126:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575134.071138:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575134.071144:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616479104:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788575134.071154:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d880 x1875449616479232/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575134.071162:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575134.071164:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788575134.071166:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575134.071169:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7046 00000100:00080000:1.0:1788575134.071171:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575134.071174:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575134.071178:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575134.071186:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ce00 x1875449616479360/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575134.071209:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575134.071243:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575134/real 1788575134] req@ffff9ada4566e680 x1875449616479104/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575159 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575134.071257:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e680 x1875449616479104/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575159 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575134.071266:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575134.071269:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575134.071272:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616479104:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575134.071299:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616479232:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575134.071322:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575134.071335:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575134/real 1788575134] req@ffff9ada4566d880 x1875449616479232/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575159 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575134.071348:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d880 x1875449616479232/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575159 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575134.071356:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575134.071359:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575134.071363:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616479232:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575134.071369:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616479360:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575134.071388:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575134.071400:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575134/real 1788575134] req@ffff9ada4566ce00 x1875449616479360/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575159 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575134.071411:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ce00 x1875449616479360/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575159 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575134.071419:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575134.071423:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575134.071426:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616479360:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575134.262989:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575134.582994:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575135.286971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575135.607105:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575136.311019:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575136.630985:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575137.334948:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575137.655028:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575138.358957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575138.679007:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575139.190968:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575139.190983:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575139.190989:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575139.191001:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7051 00000100:00080000:1.0:1788575139.191014:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575139.191023:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575139.191067:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c700 x1875449616479488/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575139.191099:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575139.191103:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575139.191105:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575139.191110:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7051 00000100:00080000:1.0:1788575139.191113:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575139.191118:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575139.191142:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575139.191157:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616479488:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788575139.191160:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c380 x1875449616479616/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575139.191187:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575139.191192:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788575139.191195:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575139.191200:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7051 00000100:00080000:1.0:1788575139.191207:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575139.191212:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575139.191218:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575139.191231:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f100 x1875449616479744/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575139.191243:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575139.191295:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575139/real 1788575139] req@ffff9ada4566c700 x1875449616479488/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575164 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575139.191313:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c700 x1875449616479488/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575164 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575139.191325:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575139.191329:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575139.191333:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616479488:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575139.191356:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616479616:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575139.191379:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575139.191392:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575139/real 1788575139] req@ffff9ada4566c380 x1875449616479616/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575164 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575139.191404:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c380 x1875449616479616/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575164 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575139.191412:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575139.191415:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575139.191419:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616479616:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575139.191425:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616479744:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575139.191444:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575139.191454:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575139/real 1788575139] req@ffff9ada4566f100 x1875449616479744/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575164 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575139.191464:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f100 x1875449616479744/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575164 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575139.191472:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575139.191475:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575139.191478:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616479744:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575139.382969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575139.703050:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575140.406987:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575140.727079:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575141.430947:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575141.751046:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575142.454979:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575142.774988:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575143.478948:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575143.798985:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575144.310962:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575144.310976:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575144.310979:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575144.310987:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7056 00000100:00080000:2.0:1788575144.310994:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575144.311000:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575144.311029:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616479872/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575144.311055:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575144.311058:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575144.311059:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575144.311063:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7056 00000100:00080000:2.0:1788575144.311066:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575144.311070:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575144.311080:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575144.311088:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616479872:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788575144.311095:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616480000/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575144.311103:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575144.311105:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575144.311106:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575144.311108:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7056 00000100:00080000:2.0:1788575144.311111:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575144.311113:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575144.311117:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575144.311125:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616480128/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575144.311138:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575144.311164:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575144/real 1788575144] req@ffff9ada60626a00 x1875449616479872/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575169 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575144.311175:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616479872/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575169 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575144.311182:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575144.311184:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575144.311187:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616479872:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575144.311203:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616480000:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575144.311214:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575144.311220:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575144/real 1788575144] req@ffff9ada60624380 x1875449616480000/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575169 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575144.311228:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616480000/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575169 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575144.311232:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575144.311234:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575144.311235:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616480000:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575144.311238:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616480128:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575144.311258:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575144.311263:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575144/real 1788575144] req@ffff9ada60626680 x1875449616480128/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575169 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575144.311270:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616480128/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575169 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575144.311274:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575144.311276:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575144.311278:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616480128:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575144.502944:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575144.822999:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575145.526929:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575145.847021:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575146.550951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575146.870986:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575147.574946:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575147.895039:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575148.598967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575148.919070:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575149.430971:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575149.430982:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575149.430987:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575149.430997:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7061 00000100:00080000:2.0:1788575149.431007:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575149.431015:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575149.431053:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627100 x1875449616480256/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575149.431084:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575149.431088:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575149.431090:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575149.431095:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7061 00000100:00080000:2.0:1788575149.431099:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575149.431105:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575149.431116:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575149.431131:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624700 x1875449616480384/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575149.431145:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616480256:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788575149.431150:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575149.431176:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575149.431180:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575149.431187:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7061 00000100:00080000:2.0:1788575149.431193:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575149.431201:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575149.431207:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575149.431223:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616480512/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575149.431263:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575149.431307:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575149/real 1788575149] req@ffff9ada60627100 x1875449616480256/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575174 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575149.431325:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627100 x1875449616480256/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575174 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575149.431335:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575149.431338:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575149.431342:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616480256:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575149.431367:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616480384:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575149.431386:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575149.431400:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575149/real 1788575149] req@ffff9ada60624700 x1875449616480384/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575174 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575149.431414:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624700 x1875449616480384/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575174 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575149.431422:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575149.431425:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575149.431430:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616480384:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575149.431436:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616480512:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575149.431458:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575149.431470:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575149/real 1788575149] req@ffff9ada60626680 x1875449616480512/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575174 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575149.431483:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616480512/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575174 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575149.431491:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575149.431494:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575149.431497:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616480512:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575149.622939:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575149.943050:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575150.646985:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575150.967102:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575151.670950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575151.991030:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575152.694934:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575153.015047:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575153.718978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575154.038977:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575154.550978:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575154.550986:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575154.550989:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575154.550998:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7067 00000100:00080000:1.0:1788575154.551008:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575154.551016:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575154.551053:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d180 x1875449616480640/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575154.551080:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575154.551084:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575154.551086:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575154.551090:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7067 00000100:00080000:1.0:1788575154.551094:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575154.551097:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575154.551105:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575154.551116:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ca80 x1875449616480768/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575154.551124:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575154.551127:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788575154.551128:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575154.551134:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7067 00000100:00100000:3.0:1788575154.551134:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616480640:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788575154.551142:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575154.551146:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575154.551149:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575154.551158:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d500 x1875449616480896/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575154.551203:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575154.551250:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575154/real 1788575154] req@ffff9ada4566d180 x1875449616480640/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575179 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575154.551276:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d180 x1875449616480640/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575179 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575154.551288:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575154.551293:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575154.551297:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616480640:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575154.551322:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616480768:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575154.551354:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575154.551368:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575154/real 1788575154] req@ffff9ada4566ca80 x1875449616480768/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575179 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575154.551382:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ca80 x1875449616480768/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575179 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575154.551391:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575154.551395:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575154.551398:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616480768:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575154.551405:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616480896:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575154.551425:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575154.551437:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575154/real 1788575154] req@ffff9ada4566d500 x1875449616480896/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575179 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575154.551450:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d500 x1875449616480896/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575179 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575154.551459:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575154.551462:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575154.551466:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616480896:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575154.742952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575155.063041:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575155.766958:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575156.087019:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575156.790995:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575157.111081:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575157.814965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575158.135005:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575158.838916:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575159.158979:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575159.671002:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575159.671010:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575159.671014:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575159.671023:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7072 00000100:00080000:0.0:1788575159.671032:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575159.671039:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575159.671070:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616481024/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575159.671091:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575159.671094:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575159.671096:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575159.671098:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7072 00000100:00080000:0.0:1788575159.671100:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575159.671103:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575159.671110:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575159.671117:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616481152/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575159.671125:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575159.671128:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575159.671130:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575159.671132:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7072 00000100:00080000:0.0:1788575159.671134:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575159.671137:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575159.671141:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575159.671146:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616481024:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575159.671151:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616481280/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575159.671198:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575159.671227:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575159/real 1788575159] req@ffff9ada4557dc00 x1875449616481024/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575184 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575159.671241:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616481024/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575184 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575159.671249:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575159.671252:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575159.671260:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616481024:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575159.671265:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616481152:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575159.671280:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575159.671289:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575159/real 1788575159] req@ffff9ada4557d880 x1875449616481152/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575184 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575159.671299:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616481152/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575184 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575159.671308:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575159.671312:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575159.671315:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616481152:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575159.671355:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616481280:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575159.671373:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575159.671385:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575159/real 1788575159] req@ffff9ada4557ca80 x1875449616481280/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575184 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575159.671398:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616481280/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575184 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575159.671406:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575159.671410:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575159.671414:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616481280:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575159.862964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575160.183029:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575160.886954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575161.207041:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575161.910974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575162.231031:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575162.934953:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575163.255056:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575163.958966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575164.278996:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575164.790950:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575164.790958:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575164.790961:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575164.790970:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7077 00000100:00080000:2.0:1788575164.790979:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575164.790984:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575164.791013:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616481408/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575164.791036:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575164.791039:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575164.791040:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575164.791042:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7077 00000100:00080000:2.0:1788575164.791045:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575164.791047:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575164.791054:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575164.791063:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616481536/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575164.791078:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1788575164.791083:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616481408:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788575164.791084:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575164.791088:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575164.791093:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7077 00000100:00080000:2.0:1788575164.791096:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575164.791098:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575164.791102:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575164.791111:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616481664/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575164.791146:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575164.791174:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575164/real 1788575164] req@ffff9ada60624380 x1875449616481408/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575189 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575164.791190:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616481408/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575189 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575164.791197:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575164.791201:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575164.791204:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616481408:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575164.791222:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616481536:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575164.791266:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575164.791280:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575164/real 1788575164] req@ffff9ada60626a00 x1875449616481536/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575189 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575164.791294:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616481536/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575189 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575164.791303:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575164.791307:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575164.791311:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616481536:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575164.791318:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616481664:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575164.791341:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575164.791351:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575164/real 1788575164] req@ffff9ada60624a80 x1875449616481664/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575189 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575164.791360:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616481664/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575189 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575164.791366:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575164.791368:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575164.791371:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616481664:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575164.982958:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575165.303021:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575166.006948:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575166.327038:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575167.030950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575167.351018:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575168.054977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575168.375065:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575169.078948:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575169.399031:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575169.910992:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575169.911001:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575169.911004:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575169.911012:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7082 00000100:00080000:2.0:1788575169.911021:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575169.911027:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575169.911059:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616481792/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575169.911085:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575169.911088:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575169.911089:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575169.911092:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7082 00000100:00080000:2.0:1788575169.911094:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575169.911096:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575169.911104:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575169.911111:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616481920/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575169.911118:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575169.911121:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575169.911122:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575169.911125:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7082 00000100:00080000:2.0:1788575169.911127:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575169.911129:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575169.911133:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575169.911139:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616482048/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575169.911187:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616481792:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788575169.911279:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575169.911336:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575169/real 1788575169] req@ffff9ada60625880 x1875449616481792/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575194 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575169.911362:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616481792/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575194 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575169.911376:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575169.911382:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575169.911386:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616481792:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575169.911395:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616481920:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575169.911423:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575169.911435:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575169/real 1788575169] req@ffff9ada60625500 x1875449616481920/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575194 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575169.911444:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616481920/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575194 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575169.911450:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575169.911453:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575169.911455:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616481920:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575169.911462:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616482048:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575169.911476:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575169.911483:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575169/real 1788575169] req@ffff9ada60625c00 x1875449616482048/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575194 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575169.911491:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616482048/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575194 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575169.911496:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575169.911507:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575169.911509:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616482048:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575170.103032:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575170.423011:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575171.126978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575171.447068:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575172.150955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575172.471023:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575173.174937:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575173.494981:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575174.198927:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575174.519035:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575175.030927:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575175.030935:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575175.030938:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575175.030946:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7087 00000100:00080000:2.0:1788575175.030955:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575175.030960:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575175.030995:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616482176/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575175.031026:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575175.031031:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575175.031033:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575175.031037:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7087 00000100:00080000:2.0:1788575175.031041:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575175.031050:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575175.031060:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575175.031075:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616482304/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575175.031075:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616482176:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788575175.031086:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575175.031092:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575175.031094:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575175.031097:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7087 00000100:00080000:2.0:1788575175.031099:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575175.031102:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575175.031107:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575175.031121:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616482432/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575175.031145:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575175.031173:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575175/real 1788575175] req@ffff9ada60625f80 x1875449616482176/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575200 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575175.031188:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616482176/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575200 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575175.031196:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575175.031198:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575175.031201:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616482176:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575175.031220:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616482304:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575175.031236:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575175.031244:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575175/real 1788575175] req@ffff9ada60624e00 x1875449616482304/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575200 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575175.031252:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616482304/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575200 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575175.031258:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575175.031260:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575175.031262:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616482304:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575175.031266:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616482432:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575175.031276:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575175.031282:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575175/real 1788575175] req@ffff9ada60627480 x1875449616482432/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575200 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575175.031289:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616482432/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575200 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575175.031295:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575175.031299:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575175.031302:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616482432:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575175.222958:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575175.543047:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575176.246951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575176.567049:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575177.271021:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575177.590982:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575178.294933:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575178.615006:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575179.318960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575179.639068:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575180.150955:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575180.150964:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575180.150968:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575180.150977:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7092 00000100:00080000:0.0:1788575180.150993:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575180.151000:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575180.151032:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616482560/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575180.151056:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575180.151059:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575180.151061:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575180.151064:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7092 00000100:00080000:0.0:1788575180.151067:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575180.151071:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575180.151079:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575180.151089:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616482560:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575180.151094:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616482688/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575180.151104:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575180.151107:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575180.151109:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575180.151112:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7092 00000100:00080000:0.0:1788575180.151115:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575180.151118:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575180.151122:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575180.151132:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ce00 x1875449616482816/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575180.151167:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616482688:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575180.151186:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616482816:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575180.342947:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575180.663041:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575181.366965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575181.687027:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575182.390938:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575182.711019:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1788575183.223199:0:2371:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:0.0:1788575183.223237:0:2371:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1788575183.223253:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:0.0:1788575183.223290:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:0.0:1788575183.223299:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:3.0:1788575183.223339:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575180/real 1788575183] req@ffff9ada4557c700 x1875449616482688/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575205 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575183.223365:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616482688/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575205 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575183.223377:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575183.223382:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575183.223386:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616482688:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575183.223394:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575180/real 1788575183] req@ffff9ada4557ce00 x1875449616482816/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575205 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575183.223407:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ce00 x1875449616482816/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575205 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575183.223414:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575183.223418:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575183.223421:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616482816:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575183.223428:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575180/real 1788575183] req@ffff9ada4557f480 x1875449616482560/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575205 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575183.223440:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616482560/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575205 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575183.223447:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575183.223450:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575183.223454:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616482560:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575183.414963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575183.734987:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575184.438943:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575184.759058:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575185.270963:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575185.270971:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575185.270974:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575185.270981:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7097 00000100:00080000:0.0:1788575185.271003:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575185.271008:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575185.271037:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c000 x1875449616482944/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575185.271061:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575185.271063:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575185.271064:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575185.271066:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7097 00000100:00080000:0.0:1788575185.271068:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575185.271070:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575185.271076:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575185.271083:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c380 x1875449616483072/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575185.271089:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575185.271092:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575185.271094:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575185.271097:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7097 00000100:00080000:0.0:1788575185.271099:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575185.271101:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575185.271104:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575185.271110:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616483200/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575185.271126:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616482944:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788575185.271211:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575185.271258:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575185/real 1788575185] req@ffff9ada4557c000 x1875449616482944/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575210 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575185.271283:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c000 x1875449616482944/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575210 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575185.271296:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575185.271301:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575185.271306:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616482944:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575185.271314:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616483072:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575185.271341:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575185.271355:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575185/real 1788575185] req@ffff9ada4557c380 x1875449616483072/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575210 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575185.271369:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c380 x1875449616483072/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575210 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575185.271378:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575185.271382:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575185.271385:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616483072:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575185.271417:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616483200:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575185.271442:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575185.271454:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575185/real 1788575185] req@ffff9ada4557f480 x1875449616483200/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575210 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575185.271467:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616483200/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575210 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575185.271474:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575185.271477:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575185.271479:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616483200:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575185.462967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575185.783016:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575186.486942:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575186.807061:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575187.510972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575187.831032:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575188.534928:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575188.855043:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575189.558947:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575189.879005:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575190.390948:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575190.390958:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575190.390962:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575190.390971:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7102 00000100:00080000:0.0:1788575190.390982:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575190.390989:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575190.391022:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ce00 x1875449616483328/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575190.391050:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575190.391053:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575190.391055:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575190.391059:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7102 00000100:00080000:0.0:1788575190.391063:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575190.391067:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575190.391077:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575190.391082:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616483328:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575190.391093:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616483456/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575190.391106:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575190.391110:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575190.391112:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575190.391116:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7102 00000100:00080000:0.0:1788575190.391120:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575190.391124:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575190.391130:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1788575190.391132:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575190.391146:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616483584/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575190.391163:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575190/real 1788575190] req@ffff9ada4557ce00 x1875449616483328/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575215 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575190.391176:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ce00 x1875449616483328/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575215 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575190.391185:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575190.391188:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575190.391191:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616483328:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575190.391209:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616483456:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575190.391222:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575190.391232:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575190/real 1788575190] req@ffff9ada4557c700 x1875449616483456/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575215 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575190.391243:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616483456/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575215 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575190.391251:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575190.391254:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575190.391257:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616483456:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575190.391262:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616483584:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575190.391280:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575190.391291:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575190/real 1788575190] req@ffff9ada4557ca80 x1875449616483584/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575215 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575190.391303:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616483584/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575215 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575190.391310:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575190.391313:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575190.391316:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616483584:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575190.582908:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575190.902982:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575191.606952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575191.927022:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575192.630940:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575192.950975:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575193.654929:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575193.975131:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575194.678982:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575194.999031:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575195.510985:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575195.510992:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575195.510996:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575195.511003:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7108 00000100:00080000:2.0:1788575195.511011:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575195.511016:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575195.511044:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616483712/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575195.511066:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575195.511069:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575195.511070:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575195.511072:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7108 00000100:00080000:2.0:1788575195.511074:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575195.511076:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575195.511083:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575195.511090:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616483840/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575195.511103:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575195.511106:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575195.511107:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575195.511109:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7108 00000100:00080000:2.0:1788575195.511111:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575195.511113:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575195.511117:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575195.511127:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616483968/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575195.511128:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616483712:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788575195.511212:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575195.511247:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575195/real 1788575195] req@ffff9ada60625180 x1875449616483712/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575220 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575195.511262:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616483712/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575220 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575195.511271:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575195.511274:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575195.511278:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616483712:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575195.511283:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616483840:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575195.511297:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575195.511304:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575195/real 1788575195] req@ffff9ada60626d80 x1875449616483840/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575220 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575195.511313:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616483840/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575220 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575195.511321:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575195.511324:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575195.511327:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616483840:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575195.511357:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616483968:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575195.511397:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575195.511406:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575195/real 1788575195] req@ffff9ada60627b80 x1875449616483968/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575220 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575195.511423:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616483968/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575220 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575195.511431:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575195.511434:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575195.511437:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616483968:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575195.702943:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575196.023051:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575196.726970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575197.047035:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575197.750926:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575198.071012:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575198.774941:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575199.095015:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575199.798962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575200.119019:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575200.630984:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575200.630992:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575200.630995:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575200.631005:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7113 00000100:00080000:0.0:1788575200.631015:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575200.631022:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575200.631055:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616484096/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575200.631088:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575200.631090:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575200.631092:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575200.631094:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7113 00000100:00080000:0.0:1788575200.631097:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575200.631099:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575200.631106:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575200.631115:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616484224/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575200.631121:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575200.631124:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575200.631125:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575200.631127:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7113 00000100:00080000:0.0:1788575200.631129:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1788575200.631133:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616484096:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788575200.631134:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575200.631139:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575200.631145:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616484352/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575200.631186:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575200.631225:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575200/real 1788575200] req@ffff9ada4557d880 x1875449616484096/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575225 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575200.631243:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616484096/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575225 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575200.631250:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575200.631252:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575200.631254:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616484096:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575200.631259:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616484224:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575200.631274:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575200.631282:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575200/real 1788575200] req@ffff9ada4557dc00 x1875449616484224/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575225 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575200.631293:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616484224/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575225 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575200.631298:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575200.631300:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575200.631302:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616484224:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575200.631319:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616484352:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575200.631328:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575200.631333:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575200/real 1788575200] req@ffff9ada4557f800 x1875449616484352/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575225 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575200.631340:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616484352/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575225 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575200.631344:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575200.631345:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575200.631347:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616484352:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788575200.822938:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575201.142985:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575201.846939:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575202.167025:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575202.870992:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575203.191034:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575203.894996:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575204.215071:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575204.918986:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575205.239043:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575205.750951:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575205.750960:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575205.750965:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575205.750975:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7118 00000100:00080000:2.0:1788575205.750985:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575205.750992:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575205.751034:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627800 x1875449616484480/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575205.751058:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575205.751062:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575205.751064:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575205.751068:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7118 00000100:00080000:2.0:1788575205.751073:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575205.751078:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575205.751093:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575205.751108:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626300 x1875449616484608/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575205.751113:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616484480:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788575205.751126:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575205.751130:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575205.751132:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575205.751137:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7118 00000100:00080000:2.0:1788575205.751141:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575205.751145:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575205.751151:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575205.751165:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616484736/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575205.751186:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575205.751223:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575205/real 1788575205] req@ffff9ada60627800 x1875449616484480/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575230 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575205.751244:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627800 x1875449616484480/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575230 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575205.751262:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575205.751266:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575205.751270:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616484480:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575205.751293:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616484608:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575205.751317:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575205.751330:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575205/real 1788575205] req@ffff9ada60626300 x1875449616484608/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575230 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575205.751356:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626300 x1875449616484608/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575230 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575205.751366:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575205.751369:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575205.751372:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616484608:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575205.751379:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616484736:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575205.751400:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575205.751412:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575205/real 1788575205] req@ffff9ada60627b80 x1875449616484736/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575230 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575205.751424:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616484736/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575230 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575205.751433:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575205.751437:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575205.751440:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616484736:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788575205.942967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575206.263013:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575206.967010:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575207.287055:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575207.990911:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575208.311122:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575209.014955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575209.335022:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575210.038959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575210.359060:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575210.870978:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575210.870988:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575210.870992:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575210.871000:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7123 00000100:00080000:1.0:1788575210.871010:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575210.871016:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575210.871050:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ea00 x1875449616484864/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575210.871073:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575210.871075:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575210.871080:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575210.871083:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7123 00000100:00080000:1.0:1788575210.871086:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575210.871089:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575210.871097:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575210.871106:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f480 x1875449616484992/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575210.871115:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575210.871118:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1788575210.871118:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616484864:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788575210.871122:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575210.871129:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7123 00000100:00080000:1.0:1788575210.871131:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575210.871133:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575210.871136:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575210.871146:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e300 x1875449616485120/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575210.871180:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575210.871209:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575210/real 1788575210] req@ffff9ada4566ea00 x1875449616484864/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575235 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575210.871224:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ea00 x1875449616484864/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575235 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575210.871232:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575210.871235:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575210.871237:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616484864:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575210.871256:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616484992:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575210.871271:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575210.871278:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575210/real 1788575210] req@ffff9ada4566f480 x1875449616484992/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575235 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575210.871286:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f480 x1875449616484992/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575235 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575210.871292:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575210.871294:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575210.871296:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616484992:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575210.871299:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616485120:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575210.871309:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575210.871315:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575210/real 1788575210] req@ffff9ada4566e300 x1875449616485120/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575235 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575210.871322:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e300 x1875449616485120/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575235 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575210.871327:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575210.871329:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575210.871331:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616485120:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788575211.062957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575211.382996:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575212.086963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575212.407018:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575213.110961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575213.431003:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575214.134955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575214.455081:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575215.158965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575215.479188:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575215.990968:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575215.990979:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575215.990983:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575215.990993:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7128 00000100:00080000:1.0:1788575215.991004:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575215.991012:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575215.991048:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f800 x1875449616485248/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575215.991078:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575215.991083:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575215.991085:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575215.991090:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7128 00000100:00080000:1.0:1788575215.991094:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575215.991099:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575215.991109:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575215.991126:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616485248:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788575215.991127:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566dc00 x1875449616485376/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575215.991154:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575215.991158:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788575215.991160:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575215.991165:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7128 00000100:00080000:1.0:1788575215.991169:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575215.991174:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575215.991179:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575215.991192:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ed80 x1875449616485504/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575215.991202:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575215.991238:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575215/real 1788575215] req@ffff9ada4566f800 x1875449616485248/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575240 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575215.991254:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f800 x1875449616485248/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575240 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575215.991262:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575215.991265:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575215.991268:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616485248:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575215.991286:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616485376:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575215.991302:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575215.991310:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575215/real 1788575215] req@ffff9ada4566dc00 x1875449616485376/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575240 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575215.991319:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566dc00 x1875449616485376/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575240 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575215.991325:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575215.991327:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575215.991329:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616485376:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575215.991333:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616485504:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575215.991343:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575215.991357:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575215/real 1788575215] req@ffff9ada4566ed80 x1875449616485504/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575240 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575215.991365:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ed80 x1875449616485504/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575240 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575215.991370:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575215.991372:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575215.991374:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616485504:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788575216.182975:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575216.503031:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575217.206944:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575217.527036:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575218.230968:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575218.551026:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575219.254962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575219.575047:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575220.278961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575220.599007:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788575221.111009:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575221.111019:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788575221.111024:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575221.111035:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7133 00000100:00080000:3.0:1788575221.111047:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575221.111055:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575221.111095:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3100 x1875449616485632/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575221.111136:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575221.111139:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788575221.111141:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575221.111146:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7133 00000100:00080000:3.0:1788575221.111150:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575221.111153:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575221.111164:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575221.111179:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2680 x1875449616485760/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788575221.111185:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616485632:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788575221.111199:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575221.111203:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788575221.111206:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575221.111211:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7133 00000100:00080000:3.0:1788575221.111233:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575221.111238:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575221.111244:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575221.111257:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1f80 x1875449616485888/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575221.111275:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575221.111314:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575221/real 1788575221] req@ffff9ada604e3100 x1875449616485632/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575246 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575221.111334:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3100 x1875449616485632/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575246 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575221.111345:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575221.111350:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575221.111353:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616485632:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575221.111377:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616485760:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575221.111402:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575221.111417:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575221/real 1788575221] req@ffff9ada604e2680 x1875449616485760/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575246 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575221.111431:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2680 x1875449616485760/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575246 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575221.111439:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575221.111441:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575221.111444:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616485760:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575221.111448:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616485888:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575221.111464:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575221.111471:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575221/real 1788575221] req@ffff9ada604e1f80 x1875449616485888/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575246 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575221.111479:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1f80 x1875449616485888/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575246 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575221.111484:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575221.111486:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575221.111488:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616485888:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788575221.302960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575221.623058:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575222.326980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575222.646986:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575223.351052:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575223.671053:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575224.374980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575224.694995:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575225.398930:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575225.719050:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788575226.231011:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575226.231023:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788575226.231028:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575226.231039:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7138 00000100:00080000:3.0:1788575226.231052:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575226.231061:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575226.231111:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3b80 x1875449616486016/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575226.231146:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575226.231150:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788575226.231152:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575226.231157:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7138 00000100:00080000:3.0:1788575226.231162:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575226.231166:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575226.231178:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575226.231197:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1c00 x1875449616486144/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575226.231207:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575226.231210:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788575226.231212:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575226.231215:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7138 00000100:00080000:3.0:1788575226.231218:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575226.231221:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575226.231225:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575226.231234:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0000 x1875449616486272/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788575226.231256:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616486016:192.168.202.113@tcp:250:kworker.0 00000800:00000100:0.0:1788575226.231344:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575226.231385:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575226/real 1788575226] req@ffff9ada604e3b80 x1875449616486016/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575251 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575226.231400:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3b80 x1875449616486016/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575251 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575226.231409:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575226.231412:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575226.231415:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616486016:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575226.231421:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616486144:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575226.231437:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575226.231445:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575226/real 1788575226] req@ffff9ada604e1c00 x1875449616486144/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575251 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575226.231454:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1c00 x1875449616486144/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575251 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575226.231459:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575226.231461:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575226.231463:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616486144:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575226.231485:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616486272:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575226.231497:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575226.231510:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575226/real 1788575226] req@ffff9ada604e0000 x1875449616486272/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575251 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575226.231523:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0000 x1875449616486272/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575251 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575226.231532:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575226.231535:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575226.231539:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616486272:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788575226.422977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575226.743091:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575227.446968:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575227.767029:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575228.470944:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575228.791099:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575229.494936:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575229.815006:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575230.518949:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575230.838990:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575231.350965:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575231.350975:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575231.350979:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575231.350988:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7143 00000100:00080000:2.0:1788575231.350998:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575231.351005:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575231.351040:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616486400/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575231.351069:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575231.351073:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575231.351079:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575231.351083:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7143 00000100:00080000:2.0:1788575231.351087:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575231.351091:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575231.351100:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575231.351112:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616486528/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788575231.351117:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616486400:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788575231.351132:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575231.351135:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575231.351137:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575231.351142:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7143 00000100:00080000:2.0:1788575231.351145:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575231.351149:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575231.351154:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575231.351165:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616486656/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575231.351177:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575231.351205:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575231/real 1788575231] req@ffff9ada60626d80 x1875449616486400/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575256 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575231.351218:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616486400/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575256 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575231.351226:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575231.351229:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575231.351232:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616486400:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575231.351252:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616486528:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575231.351276:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575231.351289:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575231/real 1788575231] req@ffff9ada60625180 x1875449616486528/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575256 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575231.351302:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616486528/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575256 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575231.351309:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575231.351312:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575231.351314:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616486528:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575231.351319:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616486656:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575231.351334:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575231.351341:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575231/real 1788575231] req@ffff9ada60627480 x1875449616486656/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575256 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575231.351348:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616486656/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575256 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575231.351354:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575231.351356:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575231.351358:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616486656:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788575231.542980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575231.863003:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575232.566941:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575232.887046:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575233.590981:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575233.911023:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575234.614928:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575234.935083:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575235.638984:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575235.959089:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575236.470994:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575236.471004:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575236.471009:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575236.471019:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7149 00000100:00080000:0.0:1788575236.471030:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575236.471038:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575236.471075:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616486784/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575236.471106:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575236.471110:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575236.471112:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575236.471117:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7149 00000100:00080000:0.0:1788575236.471120:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575236.471125:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575236.471136:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575236.471151:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616486912/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575236.471159:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616486784:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788575236.471167:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575236.471172:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575236.471175:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575236.471180:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7149 00000100:00080000:0.0:1788575236.471184:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575236.471188:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575236.471195:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575236.471209:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616487040/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575236.471234:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575236.471288:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575236/real 1788575236] req@ffff9ada4557e300 x1875449616486784/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575261 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575236.471305:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616486784/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575261 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575236.471314:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575236.471317:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575236.471320:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616486784:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575236.471340:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616486912:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575236.471358:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575236.471366:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575236/real 1788575236] req@ffff9ada4557fb80 x1875449616486912/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575261 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575236.471375:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616486912/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575261 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575236.471381:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575236.471383:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575236.471385:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616486912:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575236.471390:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616487040:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575236.471400:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575236.471406:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575236/real 1788575236] req@ffff9ada4557ed80 x1875449616487040/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575261 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575236.471414:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616487040/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575261 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575236.471419:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575236.471421:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575236.471423:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616487040:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788575236.662933:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575236.983305:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575237.686954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575238.006980:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575238.710944:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575239.031017:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575239.735132:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575240.054994:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575240.758982:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575241.079059:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575241.590966:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575241.590977:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575241.590981:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575241.590993:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7154 00000100:00080000:0.0:1788575241.591004:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575241.591012:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575241.591051:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616487168/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575241.591087:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575241.591091:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575241.591093:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575241.591097:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7154 00000100:00080000:0.0:1788575241.591102:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575241.591106:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575241.591116:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575241.591132:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616487296/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575241.591143:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616487168:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788575241.591149:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575241.591154:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575241.591156:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575241.591161:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7154 00000100:00080000:0.0:1788575241.591166:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575241.591171:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575241.591176:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575241.591190:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557df80 x1875449616487424/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575241.591215:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575241.591253:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575241/real 1788575241] req@ffff9ada4557ea00 x1875449616487168/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575266 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575241.591273:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616487168/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575266 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575241.591285:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575241.591289:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575241.591293:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616487168:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575241.591317:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616487296:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575241.591343:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575241.591357:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575241/real 1788575241] req@ffff9ada4557e680 x1875449616487296/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575266 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575241.591403:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616487296/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575266 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575241.591414:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575241.591418:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575241.591421:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616487296:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575241.591428:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616487424:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575241.591450:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575241.591463:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575241/real 1788575241] req@ffff9ada4557df80 x1875449616487424/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575266 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575241.591477:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557df80 x1875449616487424/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575266 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575241.591485:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575241.591488:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575241.591492:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616487424:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788575241.782968:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575242.103004:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575242.806958:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575243.127045:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575243.830978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575244.151063:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575244.854969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575245.175044:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575245.878947:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575246.199030:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575246.710960:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575246.710967:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575246.710970:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575246.710979:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7159 00000100:00080000:0.0:1788575246.710988:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575246.710994:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575246.711026:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d180 x1875449616487552/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575246.711048:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575246.711051:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575246.711052:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575246.711055:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7159 00000100:00080000:0.0:1788575246.711057:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575246.711059:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575246.711067:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575246.711082:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d500 x1875449616487680/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575246.711091:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616487552:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788575246.711093:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575246.711100:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575246.711102:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575246.711105:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7159 00000100:00080000:0.0:1788575246.711107:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575246.711109:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575246.711113:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575246.711122:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557df80 x1875449616487808/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575246.711199:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616487680:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575246.711224:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616487808:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788575246.902945:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575247.223027:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575247.926964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575248.247020:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575248.950983:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575249.271017:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1788575249.783133:0:2370:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:2.0:1788575249.783179:0:2370:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1788575249.783196:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788575249.783234:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788575249.783248:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:3.0:1788575249.783291:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575246/real 1788575249] req@ffff9ada4557d500 x1875449616487680/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575271 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575249.783325:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d500 x1875449616487680/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575271 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575249.783339:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575249.783344:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575249.783348:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616487680:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575249.783357:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575246/real 1788575249] req@ffff9ada4557df80 x1875449616487808/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575271 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575249.783370:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557df80 x1875449616487808/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575271 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575249.783378:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575249.783381:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575249.783385:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616487808:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575249.783392:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575246/real 1788575249] req@ffff9ada4557d180 x1875449616487552/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575271 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575249.783403:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d180 x1875449616487552/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575271 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575249.783410:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575249.783414:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575249.783417:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616487552:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788575249.974968:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575250.295047:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575250.998970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575251.319045:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788575251.831002:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575251.831009:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788575251.831013:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575251.831021:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7164 00000100:00080000:3.0:1788575251.831031:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575251.831037:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575251.831068:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2300 x1875449616487936/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575251.831101:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575251.831106:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788575251.831108:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575251.831112:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7164 00000100:00080000:3.0:1788575251.831116:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575251.831119:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575251.831130:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575251.831143:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0a80 x1875449616488064/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788575251.831153:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616487936:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788575251.831153:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575251.831157:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788575251.831159:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575251.831162:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7164 00000100:00080000:3.0:1788575251.831166:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575251.831170:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575251.831174:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575251.831184:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2d80 x1875449616488192/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575251.831217:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575251.831251:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575251/real 1788575251] req@ffff9ada604e2300 x1875449616487936/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575276 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575251.831270:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2300 x1875449616487936/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575276 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575251.831280:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575251.831284:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575251.831288:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616487936:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575251.831332:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616488064:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575251.831359:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575251.831373:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575251/real 1788575251] req@ffff9ada604e0a80 x1875449616488064/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575276 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575251.831387:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0a80 x1875449616488064/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575276 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575251.831396:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575251.831399:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575251.831402:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616488064:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575251.831408:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616488192:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575251.831428:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575251.831440:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575251/real 1788575251] req@ffff9ada604e2d80 x1875449616488192/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575276 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575251.831452:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2d80 x1875449616488192/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575276 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575251.831461:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575251.831464:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575251.831468:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616488192:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788575252.022948:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575252.343051:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575253.047012:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575253.367034:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575254.070948:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575254.391041:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575255.094950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575255.415087:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575256.118937:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575256.439022:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575256.950999:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575256.951008:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575256.951012:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575256.951023:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7169 00000100:00080000:0.0:1788575256.951035:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575256.951042:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575256.951079:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616488320/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575256.951104:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575256.951108:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575256.951109:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575256.951112:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7169 00000100:00080000:0.0:1788575256.951115:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575256.951118:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575256.951125:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575256.951133:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616488448/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575256.951140:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575256.951143:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575256.951144:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575256.951147:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7169 00000100:00080000:0.0:1788575256.951149:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575256.951156:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575256.951159:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575256.951167:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616488320:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575256.951168:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616488576/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575256.951234:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575256.951271:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575256/real 1788575256] req@ffff9ada4557e680 x1875449616488320/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575281 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575256.951290:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616488320/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575281 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575256.951301:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575256.951305:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575256.951310:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616488320:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575256.951318:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616488448:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575256.951342:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575256.951355:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575256/real 1788575256] req@ffff9ada4557ea00 x1875449616488448/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575281 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575256.951364:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616488448/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575281 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575256.951370:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575256.951373:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575256.951375:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616488448:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575256.951393:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616488576:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575256.951407:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575256.951414:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575256/real 1788575256] req@ffff9ada4557ed80 x1875449616488576/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575281 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575256.951421:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616488576/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575281 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575256.951427:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575256.951429:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575256.951431:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616488576:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788575257.142966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575257.462997:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575258.166988:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575258.487043:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575259.190928:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575259.510978:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575260.214960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575260.535045:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575261.238973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575261.559029:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575262.070934:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575262.070944:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575262.070949:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575262.070960:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7174 00000100:00080000:0.0:1788575262.070972:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575262.070980:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575262.071024:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616488704/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575262.071050:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575262.071053:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575262.071054:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575262.071057:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7174 00000100:00080000:0.0:1788575262.071060:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575262.071062:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575262.071072:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575262.071081:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616488832/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575262.071088:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575262.071092:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575262.071093:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575262.071096:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7174 00000100:00080000:0.0:1788575262.071098:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575262.071100:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575262.071103:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575262.071111:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616488960/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575262.071114:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616488704:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788575262.071215:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575262.071262:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575262/real 1788575262] req@ffff9ada4557fb80 x1875449616488704/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575287 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575262.071286:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616488704/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575287 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575262.071298:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575262.071303:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575262.071307:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616488704:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575262.071316:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616488832:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575262.071342:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575262.071357:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575262/real 1788575262] req@ffff9ada4557e300 x1875449616488832/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575287 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575262.071370:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616488832/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575287 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575262.071379:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575262.071383:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575262.071387:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616488832:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575262.071420:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616488960:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575262.071443:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575262.071455:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575262/real 1788575262] req@ffff9ada4557f800 x1875449616488960/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575287 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575262.071468:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616488960/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575287 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575262.071492:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575262.071496:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575262.071500:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616488960:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788575262.262995:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575262.583049:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575263.286953:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575263.607003:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575264.310950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575264.631049:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575265.334997:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575265.655063:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575266.358947:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575266.679016:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575267.190933:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575267.190950:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575267.190955:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575267.190965:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7179 00000100:00080000:0.0:1788575267.190975:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575267.190981:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575267.191016:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616489088/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575267.191050:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575267.191054:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575267.191056:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575267.191061:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7179 00000100:00080000:0.0:1788575267.191064:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575267.191068:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575267.191078:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575267.191095:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616489216/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575267.191106:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616489088:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788575267.191109:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575267.191115:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575267.191118:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575267.191123:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7179 00000100:00080000:0.0:1788575267.191128:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575267.191132:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575267.191138:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575267.191153:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616489344/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575267.191164:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575267.191194:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575267/real 1788575267] req@ffff9ada4557dc00 x1875449616489088/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575292 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575267.191212:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616489088/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575292 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575267.191220:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575267.191224:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575267.191227:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616489088:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575267.191246:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616489216:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575267.191262:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575267.191271:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575267/real 1788575267] req@ffff9ada4557d880 x1875449616489216/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575292 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575267.191281:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616489216/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575292 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575267.191287:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575267.191289:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575267.191291:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616489216:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575267.191295:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616489344:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575267.191305:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575267.191310:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575267/real 1788575267] req@ffff9ada4557ca80 x1875449616489344/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575292 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575267.191318:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616489344/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575292 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575267.191352:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575267.191355:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575267.191357:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616489344:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788575267.382957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575267.703032:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575268.406949:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575268.727053:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575269.430957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575269.751012:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575270.454995:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575270.775012:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575271.478963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575271.799060:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788575272.310962:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575272.310972:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788575272.310976:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575272.310986:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7184 00000100:00080000:3.0:1788575272.310996:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575272.311003:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575272.311038:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1880 x1875449616489472/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575272.311066:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575272.311070:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788575272.311072:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575272.311077:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7184 00000100:00080000:3.0:1788575272.311082:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575272.311087:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575272.311097:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575272.311100:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616489472:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575272.311113:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3800 x1875449616489600/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575272.311125:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575272.311129:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788575272.311131:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575272.311135:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7184 00000100:00080000:3.0:1788575272.311140:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575272.311143:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575272.311149:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000800:00000100:0.0:1788575272.311158:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575272.311176:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1180 x1875449616489728/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788575272.311207:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575272/real 1788575272] req@ffff9ada604e1880 x1875449616489472/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575297 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575272.311224:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1880 x1875449616489472/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575297 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575272.311232:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575272.311234:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575272.311237:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616489472:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575272.311255:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616489600:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575272.311271:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575272.311279:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575272/real 1788575272] req@ffff9ada604e3800 x1875449616489600/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575297 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575272.311287:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3800 x1875449616489600/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575297 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575272.311293:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575272.311296:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575272.311299:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616489600:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575272.311306:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616489728:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575272.311325:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575272.311337:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575272/real 1788575272] req@ffff9ada604e1180 x1875449616489728/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575297 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575272.311356:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1180 x1875449616489728/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575297 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575272.311366:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575272.311369:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575272.311372:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616489728:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788575272.502978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575272.823010:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575273.526932:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575273.847030:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575274.551458:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575274.871030:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575275.574965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575275.895022:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575276.598974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575276.919023:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788575277.430971:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575277.430978:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788575277.430982:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575277.430989:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7189 00000100:00080000:3.0:1788575277.430998:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575277.431019:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575277.431050:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0700 x1875449616489856/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575277.431075:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575277.431078:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788575277.431080:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575277.431090:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7189 00000100:00080000:3.0:1788575277.431093:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575277.431101:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575277.431110:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575277.431123:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616489856:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575277.431125:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3480 x1875449616489984/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575277.431145:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575277.431149:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788575277.431151:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575277.431155:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7189 00000100:00080000:3.0:1788575277.431159:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575277.431162:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575277.431166:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575277.431176:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0380 x1875449616490112/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575277.431179:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575277.431209:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575277/real 1788575277] req@ffff9ada604e0700 x1875449616489856/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575302 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575277.431222:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0700 x1875449616489856/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575302 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575277.431228:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575277.431230:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575277.431233:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616489856:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575277.431248:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616489984:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575277.431263:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575277.431274:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575277/real 1788575277] req@ffff9ada604e3480 x1875449616489984/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575302 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575277.431286:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3480 x1875449616489984/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575302 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575277.431299:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575277.431302:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575277.431305:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616489984:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575277.431311:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616490112:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575277.431330:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575277.431341:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575277/real 1788575277] req@ffff9ada604e0380 x1875449616490112/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575302 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575277.431351:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0380 x1875449616490112/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575302 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575277.431359:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575277.431362:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575277.431365:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616490112:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788575277.622973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575277.942977:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575278.646948:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575278.967069:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575279.670983:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575279.991014:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575280.694965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575281.014986:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575281.718955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575282.039046:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575282.551003:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575282.551013:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575282.551021:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575282.551032:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7195 00000100:00080000:1.0:1788575282.551043:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575282.551050:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575282.551090:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c000 x1875449616490240/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575282.551114:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575282.551117:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575282.551118:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575282.551121:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7195 00000100:00080000:1.0:1788575282.551123:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575282.551124:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575282.551134:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575282.551142:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566df80 x1875449616490368/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575282.551153:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575282.551155:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788575282.551156:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575282.551159:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7195 00000100:00080000:1.0:1788575282.551162:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575282.551165:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575282.551167:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575282.551178:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566fb80 x1875449616490496/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788575282.551180:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616490240:192.168.202.113@tcp:250:kworker.0 00000800:00000100:0.0:1788575282.551251:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575282.551281:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575282/real 1788575282] req@ffff9ada4566c000 x1875449616490240/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575307 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575282.551294:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c000 x1875449616490240/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575307 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575282.551304:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575282.551308:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575282.551312:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616490240:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575282.551320:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616490368:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575282.551342:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575282.551353:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575282/real 1788575282] req@ffff9ada4566df80 x1875449616490368/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575307 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575282.551360:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566df80 x1875449616490368/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575307 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575282.551365:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575282.551367:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575282.551369:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616490368:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575282.551387:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616490496:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575282.551400:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575282.551406:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575282/real 1788575282] req@ffff9ada4566fb80 x1875449616490496/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575307 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575282.551412:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566fb80 x1875449616490496/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575307 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575282.551417:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575282.551418:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575282.551420:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616490496:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788575282.742959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575283.062955:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575283.766956:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575284.087026:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575284.790955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575285.111022:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575285.814929:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575286.134991:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575286.838935:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575287.159005:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788575287.670955:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575287.670962:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788575287.670965:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575287.670972:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7200 00000100:00080000:3.0:1788575287.670986:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575287.670992:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575287.671021:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0380 x1875449616490624/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575287.671038:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575287.671041:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788575287.671043:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575287.671046:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7200 00000100:00080000:3.0:1788575287.671054:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575287.671059:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575287.671068:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575287.671079:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3480 x1875449616490752/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788575287.671086:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616490624:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788575287.671088:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575287.671091:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788575287.671092:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575287.671095:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7200 00000100:00080000:3.0:1788575287.671098:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575287.671101:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575287.671105:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575287.671112:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0700 x1875449616490880/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575287.671140:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575287.671169:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575287/real 1788575287] req@ffff9ada604e0380 x1875449616490624/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575312 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575287.671185:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0380 x1875449616490624/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575312 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575287.671194:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575287.671198:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575287.671201:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616490624:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575287.671220:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616490752:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575287.671237:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575287.671256:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575287/real 1788575287] req@ffff9ada604e3480 x1875449616490752/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575312 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575287.671264:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3480 x1875449616490752/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575312 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575287.671269:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575287.671271:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575287.671273:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616490752:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575287.671276:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616490880:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575287.671287:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575287.671293:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575287/real 1788575287] req@ffff9ada604e0700 x1875449616490880/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575312 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575287.671299:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0700 x1875449616490880/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575312 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575287.671307:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575287.671309:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575287.671310:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616490880:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788575287.862922:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575288.183010:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575288.886942:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575289.206983:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575289.910940:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575290.230962:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575290.934951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575291.255023:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575291.958979:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575292.278993:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788575292.790949:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575292.790956:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788575292.790960:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575292.790967:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7205 00000100:00080000:3.0:1788575292.790975:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575292.790986:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575292.791015:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1180 x1875449616491008/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575292.791042:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575292.791046:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788575292.791048:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575292.791051:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7205 00000100:00080000:3.0:1788575292.791055:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575292.791058:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575292.791066:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575292.791079:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3800 x1875449616491136/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575292.791088:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575292.791090:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788575292.791092:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1788575292.791095:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616491008:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788575292.791095:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7205 00000100:00080000:3.0:1788575292.791098:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575292.791101:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575292.791105:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575292.791113:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1880 x1875449616491264/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575292.791149:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575292.791179:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575292/real 1788575292] req@ffff9ada604e1180 x1875449616491008/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575317 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575292.791190:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1180 x1875449616491008/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575317 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575292.791197:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575292.791199:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575292.791201:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616491008:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575292.791217:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616491136:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575292.791232:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575292.791239:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575292/real 1788575292] req@ffff9ada604e3800 x1875449616491136/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575317 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575292.791249:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3800 x1875449616491136/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575317 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575292.791256:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575292.791259:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575292.791262:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616491136:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575292.791269:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616491264:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575292.791288:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575292.791298:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575292/real 1788575292] req@ffff9ada604e1880 x1875449616491264/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575317 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575292.791309:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1880 x1875449616491264/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575317 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575292.791316:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575292.791319:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575292.791322:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616491264:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788575292.982944:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575293.303058:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575294.006971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575294.327060:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575295.030952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575295.351027:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575296.054996:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575296.375053:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575297.078966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575297.399083:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788575297.911008:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575297.911018:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788575297.911023:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575297.911034:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7210 00000100:00080000:3.0:1788575297.911048:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575297.911057:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575297.911112:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2d80 x1875449616491392/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575297.911139:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575297.911143:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788575297.911145:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575297.911148:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7210 00000100:00080000:3.0:1788575297.911151:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575297.911155:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575297.911166:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575297.911176:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0a80 x1875449616491520/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575297.911184:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575297.911187:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788575297.911188:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575297.911191:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7210 00000100:00080000:3.0:1788575297.911193:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575297.911195:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575297.911199:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575297.911208:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2300 x1875449616491648/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788575297.911210:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616491392:192.168.202.113@tcp:250:kworker.0 00000800:00000100:0.0:1788575297.911315:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575297.911374:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575297/real 1788575297] req@ffff9ada604e2d80 x1875449616491392/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575322 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575297.911393:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2d80 x1875449616491392/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575322 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575297.911402:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575297.911406:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575297.911409:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616491392:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575297.911416:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616491520:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575297.911436:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575297.911445:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575297/real 1788575297] req@ffff9ada604e0a80 x1875449616491520/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575322 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575297.911455:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0a80 x1875449616491520/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575322 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575297.911461:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575297.911463:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575297.911466:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616491520:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575297.911490:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616491648:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575297.911502:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575297.911509:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575297/real 1788575297] req@ffff9ada604e2300 x1875449616491648/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575322 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575297.911517:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2300 x1875449616491648/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575322 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575297.911522:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575297.911523:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575297.911525:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616491648:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788575298.103006:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575298.423086:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575299.127010:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575299.447069:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575300.151007:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575300.471056:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575301.174973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575301.495065:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575302.198989:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575302.519051:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575303.031043:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575303.031057:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575303.031062:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575303.031072:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7215 00000100:00080000:2.0:1788575303.031092:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575303.031103:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575303.031153:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616491776/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575303.031223:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575303.031227:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575303.031230:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575303.031234:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7215 00000100:00080000:2.0:1788575303.031239:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575303.031242:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575303.031253:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575303.031268:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616491904/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788575303.031284:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616491776:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788575303.031286:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575303.031301:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575303.031305:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575303.031310:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7215 00000100:00080000:2.0:1788575303.031314:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575303.031318:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575303.031323:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575303.031336:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616492032/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575303.031374:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575303.031411:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575303/real 1788575303] req@ffff9ada60624e00 x1875449616491776/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575328 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575303.031425:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616491776/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575328 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575303.031439:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575303.031444:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575303.031448:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616491776:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575303.031469:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616491904:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575303.031485:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575303.031493:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575303/real 1788575303] req@ffff9ada60625f80 x1875449616491904/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575328 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575303.031502:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616491904/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575328 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575303.031508:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575303.031512:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575303.031514:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616491904:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575303.031518:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616492032:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575303.031528:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575303.031534:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575303/real 1788575303] req@ffff9ada60625c00 x1875449616492032/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575328 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575303.031541:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616492032/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575328 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575303.031546:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575303.031548:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575303.031550:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616492032:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788575303.222994:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575303.543078:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575304.247004:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575304.566992:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575305.270935:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575305.591183:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575306.295009:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575306.615242:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575307.318945:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575307.639089:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788575308.151019:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575308.151042:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788575308.151047:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575308.151057:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7220 00000100:00080000:3.0:1788575308.151075:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575308.151086:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575308.151138:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0000 x1875449616492160/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575308.151165:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575308.151169:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788575308.151170:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575308.151174:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7220 00000100:00080000:3.0:1788575308.151177:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575308.151180:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575308.151190:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575308.151206:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1c00 x1875449616492288/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788575308.151216:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616492160:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788575308.151218:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575308.151226:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788575308.151227:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575308.151231:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7220 00000100:00080000:3.0:1788575308.151234:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575308.151236:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575308.151240:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575308.151250:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3b80 x1875449616492416/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575308.151298:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575308.151349:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575308/real 1788575308] req@ffff9ada604e0000 x1875449616492160/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575333 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575308.151371:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0000 x1875449616492160/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575333 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575308.151384:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575308.151389:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575308.151394:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616492160:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575308.151423:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616492288:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575308.151449:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575308.151463:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575308/real 1788575308] req@ffff9ada604e1c00 x1875449616492288/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575333 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575308.151481:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1c00 x1875449616492288/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575333 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575308.151490:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575308.151493:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575308.151497:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616492288:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575308.151508:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616492416:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575308.151527:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575308.151537:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575308/real 1788575308] req@ffff9ada604e3b80 x1875449616492416/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575333 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575308.151549:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3b80 x1875449616492416/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575333 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575308.151558:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575308.151561:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575308.151565:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616492416:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788575308.342941:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575308.663228:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575309.366956:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575309.687100:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575310.390944:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575310.711038:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575311.415010:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575311.735021:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575312.438950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575312.759035:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575313.270967:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575313.270978:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575313.270984:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575313.270995:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7225 00000100:00080000:2.0:1788575313.271007:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575313.271015:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575313.271061:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616492544/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575313.271089:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575313.271093:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575313.271095:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575313.271099:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7225 00000100:00080000:2.0:1788575313.271104:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575313.271108:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575313.271121:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575313.271137:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616492672/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575313.271151:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1788575313.271153:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616492544:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788575313.271157:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575313.271162:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575313.271168:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7225 00000100:00080000:2.0:1788575313.271173:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575313.271177:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575313.271184:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575313.271198:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616492800/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788575313.271284:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616492672:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575313.271319:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616492800:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788575313.463028:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575313.783075:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575314.487021:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575314.807107:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575315.510986:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575315.831106:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1788575316.343139:0:2372:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:1.0:1788575316.343177:0:2372:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1788575316.343190:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:1.0:1788575316.343227:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:1.0:1788575316.343238:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:0.0:1788575316.343285:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575313/real 1788575316] req@ffff9ada60625880 x1875449616492672/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575338 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575316.343320:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616492672/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575338 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575316.343336:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575316.343342:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575316.343346:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616492672:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575316.343354:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575313/real 1788575316] req@ffff9ada60624a80 x1875449616492800/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575338 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575316.343365:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616492800/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575338 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575316.343379:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575316.343382:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575316.343386:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616492800:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575316.343392:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575313/real 1788575316] req@ffff9ada60625500 x1875449616492544/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575338 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575316.343404:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616492544/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575338 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575316.343412:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575316.343415:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575316.343418:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616492544:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788575316.534959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575316.855071:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575317.559007:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575317.879059:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575318.390943:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575318.390955:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575318.390960:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575318.390972:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7230 00000100:00080000:2.0:1788575318.390985:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575318.390993:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575318.391038:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616492928/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575318.391064:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575318.391067:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575318.391069:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575318.391072:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7230 00000100:00080000:2.0:1788575318.391074:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575318.391076:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575318.391086:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575318.391096:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616493056/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575318.391104:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575318.391108:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575318.391110:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575318.391115:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7230 00000100:00080000:2.0:1788575318.391119:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575318.391124:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575318.391131:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575318.391137:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616492928:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788575318.391147:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616493184/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575318.391236:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575318.391286:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575318/real 1788575318] req@ffff9ada60626a00 x1875449616492928/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575343 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575318.391309:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616492928/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575343 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575318.391322:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575318.391326:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575318.391330:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616492928:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575318.391338:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616493056:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575318.391367:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575318.391381:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575318/real 1788575318] req@ffff9ada60624380 x1875449616493056/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575343 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575318.391394:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616493056/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575343 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575318.391403:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575318.391407:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575318.391410:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616493056:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575318.391444:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616493184:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575318.391467:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575318.391480:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575318/real 1788575318] req@ffff9ada60626680 x1875449616493184/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575343 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575318.391492:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616493184/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575343 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575318.391501:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575318.391505:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575318.391509:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616493184:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788575318.582966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575318.903080:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575319.606954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575319.926962:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575320.630929:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575320.951052:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575321.654911:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575321.975000:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575322.678922:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575322.999216:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575323.510989:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575323.510996:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575323.511000:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575323.511007:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7236 00000100:00080000:0.0:1788575323.511015:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575323.511021:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575323.511053:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616493312/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575323.511080:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575323.511085:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575323.511087:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575323.511092:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7236 00000100:00080000:0.0:1788575323.511096:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575323.511100:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575323.511110:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575323.511126:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ce00 x1875449616493440/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575323.511132:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616493312:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788575323.511147:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575323.511152:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575323.511162:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575323.511168:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7236 00000100:00080000:0.0:1788575323.511173:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575323.511178:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575323.511183:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575323.511197:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616493568/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575323.511202:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575323.511246:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575323/real 1788575323] req@ffff9ada4557c700 x1875449616493312/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575348 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575323.511263:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616493312/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575348 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575323.511271:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575323.511274:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575323.511277:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616493312:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575323.511297:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616493440:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575323.511314:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575323.511322:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575323/real 1788575323] req@ffff9ada4557ce00 x1875449616493440/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575348 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575323.511330:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ce00 x1875449616493440/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575348 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575323.511336:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575323.511338:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575323.511340:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616493440:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575323.511344:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616493568:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575323.511353:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575323.511359:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575323/real 1788575323] req@ffff9ada4557f480 x1875449616493568/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575348 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575323.511366:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616493568/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575348 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575323.511376:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575323.511378:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575323.511380:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616493568:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788575323.702977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575324.023038:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575324.726922:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575325.047034:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575325.750947:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575326.071068:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575326.774923:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575327.095018:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788575327.798953:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575328.119037:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788575328.630946:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575328.630957:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788575328.630960:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575328.630968:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7241 00000100:00080000:3.0:1788575328.630977:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575328.630983:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575328.631012:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1f80 x1875449616493696/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575328.631044:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575328.631049:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788575328.631051:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575328.631055:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7241 00000100:00080000:3.0:1788575328.631059:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575328.631064:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575328.631074:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575328.631085:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2680 x1875449616493824/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788575328.631093:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616493696:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788575328.631096:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575328.631103:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788575328.631105:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575328.631108:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7241 00000100:00080000:3.0:1788575328.631110:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575328.631113:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575328.631116:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575328.631124:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3100 x1875449616493952/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575328.631157:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575328.631191:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575328/real 1788575328] req@ffff9ada604e1f80 x1875449616493696/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575353 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575328.631205:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1f80 x1875449616493696/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575353 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575328.631213:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575328.631216:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575328.631219:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616493696:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575328.631236:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616493824:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575328.631250:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575328.631258:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575328/real 1788575328] req@ffff9ada604e2680 x1875449616493824/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575353 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575328.631266:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2680 x1875449616493824/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575353 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575328.631271:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575328.631273:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575328.631275:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616493824:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575328.631278:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616493952:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575328.631289:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575328.631294:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575328/real 1788575328] req@ffff9ada604e3100 x1875449616493952/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575353 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575328.631301:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3100 x1875449616493952/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575353 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575328.631311:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575328.631313:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575328.631315:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616493952:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788575328.822952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575329.143025:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575329.846956:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575330.167135:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575330.871001:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575331.190972:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575331.894929:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575332.214992:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575332.918965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575333.239029:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575333.751012:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575333.751023:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575333.751027:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575333.751036:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7246 00000100:00080000:0.0:1788575333.751045:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575333.751051:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575333.751085:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c380 x1875449616494080/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575333.751108:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575333.751111:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575333.751112:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575333.751115:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7246 00000100:00080000:0.0:1788575333.751117:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575333.751120:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575333.751127:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575333.751136:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c000 x1875449616494208/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575333.751144:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616494080:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788575333.751149:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575333.751152:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575333.751153:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575333.751156:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7246 00000100:00080000:0.0:1788575333.751158:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575333.751160:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575333.751163:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575333.751172:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616494336/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575333.751200:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575333.751228:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575333/real 1788575333] req@ffff9ada4557c380 x1875449616494080/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575358 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575333.751243:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c380 x1875449616494080/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575358 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575333.751250:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575333.751253:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575333.751256:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616494080:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575333.751303:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616494208:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575333.751328:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575333.751342:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575333/real 1788575333] req@ffff9ada4557c000 x1875449616494208/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575358 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575333.751364:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c000 x1875449616494208/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575358 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575333.751371:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575333.751373:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575333.751376:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616494208:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575333.751380:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616494336:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575333.751396:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575333.751404:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575333/real 1788575333] req@ffff9ada4557f480 x1875449616494336/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575358 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575333.751412:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616494336/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575358 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575333.751416:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575333.751418:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575333.751420:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616494336:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575333.942958:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575334.263032:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575334.966969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575335.286993:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575335.991007:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575336.311113:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575337.015008:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575337.335027:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575338.039009:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575338.359076:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575338.870992:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575338.871003:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575338.871007:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575338.871019:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7251 00000100:00080000:2.0:1788575338.871032:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575338.871042:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575338.871093:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624700 x1875449616494464/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575338.871129:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575338.871134:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575338.871136:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575338.871141:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7251 00000100:00080000:2.0:1788575338.871146:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575338.871151:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575338.871164:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575338.871199:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627100 x1875449616494592/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575338.871210:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575338.871213:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575338.871215:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1788575338.871218:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616494464:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788575338.871219:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7251 00000100:00080000:2.0:1788575338.871224:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575338.871229:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575338.871233:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575338.871247:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616494720/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575338.871298:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575338.871340:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575338/real 1788575338] req@ffff9ada60624700 x1875449616494464/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575363 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575338.871363:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624700 x1875449616494464/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575363 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575338.871377:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575338.871381:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575338.871386:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616494464:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575338.871412:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616494592:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575338.871432:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575338.871442:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575338/real 1788575338] req@ffff9ada60627100 x1875449616494592/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575363 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575338.871451:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627100 x1875449616494592/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575363 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575338.871457:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575338.871460:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575338.871462:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616494592:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575338.871466:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616494720:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575338.871477:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575338.871482:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575338/real 1788575338] req@ffff9ada60626680 x1875449616494720/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575363 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575338.871489:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616494720/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575363 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575338.871495:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575338.871498:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575338.871501:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616494720:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575339.062992:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575339.383068:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575340.086979:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575340.407105:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575341.111007:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575341.430973:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575342.134941:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575342.455028:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575343.159001:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575343.478975:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575343.990977:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575343.990985:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575343.990989:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575343.990997:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7256 00000100:00080000:0.0:1788575343.991006:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575343.991012:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575343.991041:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ce00 x1875449616494848/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575343.991073:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575343.991076:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575343.991078:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575343.991080:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7256 00000100:00080000:0.0:1788575343.991083:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575343.991085:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575343.991093:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575343.991101:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616494976/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575343.991108:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575343.991124:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575343.991126:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575343.991129:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7256 00000100:00080000:0.0:1788575343.991132:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575343.991138:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575343.991141:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616494848:192.168.202.113@tcp:250:kworker.0 00000008:00100000:0.0:1788575343.991142:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575343.991155:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616495104/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575343.991213:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575343.991246:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575343/real 1788575343] req@ffff9ada4557ce00 x1875449616494848/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575368 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575343.991267:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ce00 x1875449616494848/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575368 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575343.991279:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575343.991283:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575343.991288:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616494848:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575343.991295:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616494976:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575343.991318:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575343.991331:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575343/real 1788575343] req@ffff9ada4557c700 x1875449616494976/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575368 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575343.991344:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616494976/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575368 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575343.991354:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575343.991358:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575343.991362:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616494976:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575343.991386:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616495104:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575343.991410:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575343.991423:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575343/real 1788575343] req@ffff9ada4557ca80 x1875449616495104/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575368 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575343.991437:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616495104/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575368 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575343.991446:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575343.991449:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575343.991453:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616495104:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575344.182928:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575344.502999:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575345.206959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575345.527022:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575346.230919:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575346.550952:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575347.254921:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575347.575022:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575348.278999:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575348.599026:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575349.110949:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575349.110956:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575349.110960:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575349.110968:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7261 00000100:00080000:0.0:1788575349.110977:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575349.110983:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575349.111016:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616495232/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575349.111051:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575349.111054:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575349.111056:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575349.111059:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7261 00000100:00080000:0.0:1788575349.111061:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575349.111064:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575349.111073:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575349.111088:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616495360/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575349.111097:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616495232:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788575349.111106:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575349.111111:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575349.111113:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575349.111118:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7261 00000100:00080000:0.0:1788575349.111123:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575349.111127:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575349.111132:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575349.111141:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616495488/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575349.111153:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575349.111183:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575349/real 1788575349] req@ffff9ada4557d880 x1875449616495232/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575374 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575349.111199:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616495232/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575374 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575349.111207:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575349.111210:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575349.111213:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616495232:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575349.111231:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616495360:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575349.111245:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575349.111253:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575349/real 1788575349] req@ffff9ada4557dc00 x1875449616495360/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575374 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575349.111261:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616495360/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575374 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575349.111266:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575349.111268:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575349.111271:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616495360:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575349.111274:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616495488:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575349.111284:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575349.111290:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575349/real 1788575349] req@ffff9ada4557f800 x1875449616495488/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575374 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575349.111297:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616495488/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575374 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575349.111302:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575349.111304:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575349.111306:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616495488:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575349.302935:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575349.623000:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575350.326962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575350.646987:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575351.350978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575351.671047:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575352.374980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575352.695104:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575353.398925:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575353.719043:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575354.230975:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575354.230983:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575354.230987:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575354.230995:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7266 00000100:00080000:0.0:1788575354.231004:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575354.231010:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575354.231040:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616495616/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575354.231065:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575354.231068:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575354.231069:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575354.231072:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7266 00000100:00080000:0.0:1788575354.231074:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575354.231076:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575354.231085:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575354.231093:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616495744/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575354.231104:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575354.231108:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575354.231110:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575354.231114:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7266 00000100:00080000:0.0:1788575354.231119:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1788575354.231121:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616495616:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788575354.231127:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575354.231135:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575354.231148:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616495872/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575354.231194:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575354.231231:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575354/real 1788575354] req@ffff9ada4557e300 x1875449616495616/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575379 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575354.231254:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616495616/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575379 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575354.231266:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575354.231270:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575354.231275:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616495616:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575354.231283:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616495744:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575354.231308:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575354.231323:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575354/real 1788575354] req@ffff9ada4557fb80 x1875449616495744/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575379 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575354.231338:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616495744/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575379 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575354.231347:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575354.231351:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575354.231354:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616495744:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575354.231379:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616495872:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575354.231403:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575354.231417:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575354/real 1788575354] req@ffff9ada4557ed80 x1875449616495872/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575379 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575354.231431:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616495872/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575379 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575354.231439:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575354.231443:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575354.231447:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616495872:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575354.422965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575354.743039:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575355.446970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575355.767008:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575356.470971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575356.790997:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575357.494973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575357.815222:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575358.518994:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575358.839061:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575359.351012:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575359.351024:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575359.351029:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575359.351040:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7271 00000100:00080000:1.0:1788575359.351053:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575359.351061:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575359.351106:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566fb80 x1875449616496000/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575359.351132:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575359.351135:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575359.351136:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575359.351139:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7271 00000100:00080000:1.0:1788575359.351142:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575359.351145:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575359.351161:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575359.351175:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566df80 x1875449616496128/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575359.351179:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616496000:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788575359.351204:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575359.351207:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788575359.351208:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575359.351213:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7271 00000100:00080000:1.0:1788575359.351217:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575359.351219:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575359.351224:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575359.351242:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c000 x1875449616496256/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575359.351281:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575359.351313:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575359/real 1788575359] req@ffff9ada4566fb80 x1875449616496000/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575384 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575359.351328:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566fb80 x1875449616496000/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575384 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575359.351337:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575359.351340:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575359.351344:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616496000:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575359.351363:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616496128:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575359.351378:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575359.351386:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575359/real 1788575359] req@ffff9ada4566df80 x1875449616496128/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575384 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575359.351395:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566df80 x1875449616496128/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575384 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575359.351400:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575359.351402:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575359.351404:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616496128:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575359.351408:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616496256:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575359.351418:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575359.351424:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575359/real 1788575359] req@ffff9ada4566c000 x1875449616496256/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575384 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575359.351431:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c000 x1875449616496256/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575384 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575359.351436:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575359.351438:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575359.351440:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616496256:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575359.542973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575359.863048:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575360.566958:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575360.886977:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575361.590937:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575361.911026:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575362.614941:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575362.935054:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575363.638923:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575363.958987:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575364.470990:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575364.470998:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575364.471002:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575364.471011:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7277 00000100:00080000:2.0:1788575364.471020:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575364.471027:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575364.471058:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616496384/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575364.471079:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575364.471082:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575364.471083:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575364.471086:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7277 00000100:00080000:2.0:1788575364.471088:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575364.471090:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575364.471097:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575364.471106:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616496512/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575364.471114:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575364.471117:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575364.471119:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575364.471125:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7277 00000100:00100000:3.0:1788575364.471126:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616496384:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788575364.471127:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575364.471133:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575364.471137:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575364.471144:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616496640/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575364.471184:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575364.471211:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575364/real 1788575364] req@ffff9ada60624380 x1875449616496384/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575389 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575364.471225:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616496384/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575389 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575364.471237:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575364.471240:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575364.471243:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616496384:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575364.471249:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616496512:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575364.471263:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575364.471270:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575364/real 1788575364] req@ffff9ada60626a00 x1875449616496512/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575389 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575364.471278:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616496512/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575389 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575364.471288:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575364.471291:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575364.471293:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616496512:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575364.471312:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616496640:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575364.471322:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575364.471328:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575364/real 1788575364] req@ffff9ada60625500 x1875449616496640/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575389 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575364.471335:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616496640/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575389 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575364.471341:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575364.471342:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575364.471344:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616496640:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575364.662972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575364.983052:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575365.686953:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575366.007008:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575366.710967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575367.031024:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575367.734971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575368.055008:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575368.758954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575369.079040:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575369.590934:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575369.590943:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575369.590946:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575369.590954:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7282 00000100:00080000:2.0:1788575369.590964:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575369.590970:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575369.591002:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616496768/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575369.591025:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575369.591028:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575369.591030:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575369.591032:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7282 00000100:00080000:2.0:1788575369.591034:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575369.591037:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575369.591045:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575369.591053:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616496896/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575369.591061:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575369.591064:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575369.591066:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575369.591069:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7282 00000100:00080000:2.0:1788575369.591072:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575369.591077:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575369.591081:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575369.591096:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616497024/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575369.591121:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616496768:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788575369.591211:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575369.591252:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575369/real 1788575369] req@ffff9ada60624a80 x1875449616496768/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575394 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575369.591270:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616496768/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575394 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575369.591280:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575369.591284:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575369.591287:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616496768:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575369.591293:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616496896:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575369.591315:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575369.591329:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575369/real 1788575369] req@ffff9ada60625880 x1875449616496896/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575394 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575369.591343:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616496896/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575394 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575369.591352:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575369.591357:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575369.591359:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616496896:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575369.591385:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616497024:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575369.591403:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575369.591411:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575369/real 1788575369] req@ffff9ada60625c00 x1875449616497024/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575394 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575369.591424:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616497024/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575394 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575369.591440:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575369.591442:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575369.591444:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616497024:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575369.782999:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575370.103028:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575370.806920:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575371.126990:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575371.830955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575372.150988:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575372.854931:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575373.175048:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575373.878968:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575374.199009:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575374.710974:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575374.710982:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575374.710985:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575374.710993:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7287 00000100:00080000:2.0:1788575374.711002:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575374.711009:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575374.711039:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616497152/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575374.711062:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575374.711065:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575374.711066:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575374.711068:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7287 00000100:00080000:2.0:1788575374.711071:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575374.711073:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575374.711081:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575374.711091:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616497280/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575374.711107:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575374.711109:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575374.711110:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575374.711115:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7287 00000100:00080000:2.0:1788575374.711119:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575374.711126:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575374.711126:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616497152:192.168.202.113@tcp:250:kworker.0 00000008:00100000:2.0:1788575374.711139:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575374.711148:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616497408/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575374.711238:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575374.711291:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575374/real 1788575374] req@ffff9ada60625f80 x1875449616497152/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575399 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575374.711310:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616497152/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575399 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575374.711319:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575374.711323:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575374.711327:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616497152:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575374.711355:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616497280:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575374.711378:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575374.711392:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575374/real 1788575374] req@ffff9ada60624e00 x1875449616497280/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575399 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575374.711406:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616497280/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575399 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575374.711416:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575374.711420:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575374.711423:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616497280:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575374.711431:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616497408:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575374.711452:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575374.711461:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575374/real 1788575374] req@ffff9ada60627480 x1875449616497408/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575399 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575374.711470:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616497408/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575399 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575374.711489:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575374.711497:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575374.711501:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616497408:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575374.903042:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575375.223016:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575375.927011:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575376.247025:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575376.950952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575377.271025:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575377.974980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575378.295017:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575378.998985:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575379.319048:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575379.831012:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575379.831021:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575379.831032:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575379.831042:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7292 00000100:00080000:0.0:1788575379.831053:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575379.831061:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575379.831101:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616497536/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575379.831125:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575379.831128:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575379.831129:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575379.831133:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7292 00000100:00080000:0.0:1788575379.831136:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575379.831138:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575379.831153:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575379.831163:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616497664/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575379.831172:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575379.831174:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575379.831178:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1788575379.831179:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616497536:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788575379.831185:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7292 00000100:00080000:0.0:1788575379.831191:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575379.831195:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575379.831202:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575379.831227:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d180 x1875449616497792/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575379.831274:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616497664:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575379.831317:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616497792:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575380.022961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575380.343052:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575381.046943:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575381.367023:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575382.070988:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575382.390977:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1788575382.903132:0:2373:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:2.0:1788575382.903178:0:2373:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1788575382.903196:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788575382.903242:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788575382.903253:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:3.0:1788575382.903304:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575379/real 1788575382] req@ffff9ada4557e680 x1875449616497664/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575404 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575382.903332:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616497664/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575404 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575382.903346:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575382.903351:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575382.903356:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616497664:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575382.903364:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575379/real 1788575382] req@ffff9ada4557d180 x1875449616497792/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575404 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575382.903378:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d180 x1875449616497792/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575404 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575382.903387:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575382.903391:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575382.903394:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616497792:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575382.903401:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575379/real 1788575382] req@ffff9ada4557ea00 x1875449616497536/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575404 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575382.903416:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616497536/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575404 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575382.903424:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575382.903429:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575382.903432:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616497536:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575383.094941:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575383.415039:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575384.118938:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575384.439050:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575384.950928:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575384.950936:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575384.950939:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575384.950948:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7297 00000100:00080000:0.0:1788575384.950956:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575384.950963:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575384.950993:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557df80 x1875449616497920/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575384.951017:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575384.951021:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575384.951023:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575384.951027:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7297 00000100:00080000:0.0:1788575384.951031:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575384.951034:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575384.951043:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575384.951056:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d500 x1875449616498048/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575384.951067:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575384.951070:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575384.951072:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575384.951075:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7297 00000100:00100000:3.0:1788575384.951077:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616497920:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788575384.951079:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575384.951082:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575384.951086:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575384.951097:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616498176/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575384.951137:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575384.951180:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575384/real 1788575384] req@ffff9ada4557df80 x1875449616497920/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575409 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575384.951200:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557df80 x1875449616497920/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575409 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575384.951211:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575384.951215:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575384.951219:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616497920:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575384.951228:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616498048:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575384.951254:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575384.951267:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575384/real 1788575384] req@ffff9ada4557d500 x1875449616498048/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575409 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575384.951280:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d500 x1875449616498048/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575409 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575384.951289:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575384.951292:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575384.951305:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616498048:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575384.951330:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616498176:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575384.951352:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575384.951364:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575384/real 1788575384] req@ffff9ada4557ea00 x1875449616498176/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575409 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575384.951375:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616498176/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575409 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575384.951383:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575384.951386:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575384.951389:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616498176:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575385.142963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575385.463016:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575386.166958:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575386.487092:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575387.190992:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575387.511076:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575388.214949:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575388.535083:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575389.238929:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575389.559038:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575390.071004:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575390.071012:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575390.071016:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575390.071024:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7302 00000100:00080000:1.0:1788575390.071034:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575390.071043:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575390.071085:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ed80 x1875449616498304/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575390.071110:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575390.071115:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575390.071117:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575390.071122:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7302 00000100:00080000:1.0:1788575390.071126:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1788575390.071133:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616498304:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788575390.071134:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575390.071149:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575390.071167:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566dc00 x1875449616498432/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575390.071178:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575390.071181:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788575390.071183:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575390.071187:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7302 00000100:00080000:1.0:1788575390.071191:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1788575390.071194:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00080000:1.0:1788575390.071199:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575390.071205:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575390.071218:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f800 x1875449616498560/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575390.071226:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575390/real 1788575390] req@ffff9ada4566ed80 x1875449616498304/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575415 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575390.071242:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ed80 x1875449616498304/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575415 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575390.071252:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575390.071256:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575390.071259:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616498304:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575390.071278:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616498432:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575390.071293:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575390.071300:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575390/real 1788575390] req@ffff9ada4566dc00 x1875449616498432/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575415 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575390.071308:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566dc00 x1875449616498432/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575415 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575390.071314:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575390.071316:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575390.071318:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616498432:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575390.071321:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616498560:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575390.071338:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575390.071345:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575390/real 1788575390] req@ffff9ada4566f800 x1875449616498560/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575415 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575390.071352:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f800 x1875449616498560/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575415 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575390.071357:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575390.071359:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575390.071361:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616498560:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575390.262933:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575390.583085:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575391.286961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575391.607049:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575392.310925:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575392.631056:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575393.334968:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575393.655017:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575394.359029:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575394.679002:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575395.191036:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575395.191050:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575395.191055:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575395.191067:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7307 00000100:00080000:2.0:1788575395.191091:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575395.191100:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575395.191150:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616498688/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575395.191177:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575395.191181:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575395.191182:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575395.191185:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7307 00000100:00080000:2.0:1788575395.191188:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575395.191190:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575395.191200:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575395.191210:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616498816/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575395.191218:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575395.191221:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575395.191222:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575395.191225:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7307 00000100:00080000:2.0:1788575395.191227:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575395.191231:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575395.191234:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575395.191234:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616498688:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788575395.191246:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616498944/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575395.191316:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575395.191371:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575395/real 1788575395] req@ffff9ada60625180 x1875449616498688/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575420 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575395.191390:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616498688/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575420 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575395.191413:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575395.191419:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575395.191423:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616498688:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575395.191432:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616498816:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575395.191459:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575395.191471:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575395/real 1788575395] req@ffff9ada60626d80 x1875449616498816/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575420 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575395.191480:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616498816/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575420 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575395.191486:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575395.191488:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575395.191490:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616498816:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575395.191515:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616498944:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575395.191529:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575395.191537:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575395/real 1788575395] req@ffff9ada60627b80 x1875449616498944/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575420 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575395.191549:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616498944/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575420 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575395.191555:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575395.191557:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575395.191559:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616498944:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575395.382983:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575395.703062:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575396.406970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575396.726994:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575397.430956:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575397.751005:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575398.454961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575398.775005:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575399.478947:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575399.799021:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575400.310982:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575400.310995:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575400.311000:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575400.311011:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7312 00000100:00080000:2.0:1788575400.311024:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575400.311033:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575400.311083:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626300 x1875449616499072/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575400.311112:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575400.311116:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575400.311117:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575400.311121:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7312 00000100:00080000:2.0:1788575400.311129:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575400.311134:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575400.311145:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575400.311155:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616499072:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788575400.311159:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627800 x1875449616499200/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575400.311170:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575400.311173:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575400.311175:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575400.311179:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7312 00000100:00080000:2.0:1788575400.311182:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575400.311186:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575400.311190:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575400.311203:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616499328/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575400.311221:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575400.311262:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575400/real 1788575400] req@ffff9ada60626300 x1875449616499072/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575425 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575400.311278:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626300 x1875449616499072/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575425 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575400.311286:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575400.311302:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575400.311306:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616499072:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575400.311332:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616499200:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575400.311349:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575400.311357:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575400/real 1788575400] req@ffff9ada60627800 x1875449616499200/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575425 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575400.311371:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627800 x1875449616499200/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575425 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575400.311379:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575400.311381:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575400.311383:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616499200:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575400.311387:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616499328:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575400.311398:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575400.311404:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575400/real 1788575400] req@ffff9ada60627b80 x1875449616499328/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575425 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575400.311411:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616499328/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575425 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575400.311416:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575400.311417:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575400.311419:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616499328:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575400.502963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575400.823035:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575401.526950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575401.847062:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575402.550926:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575402.871017:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575403.575008:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575403.895116:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575404.598949:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575404.918993:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575405.430950:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575405.430961:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575405.430965:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575405.430974:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7317 00000100:00080000:2.0:1788575405.430985:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575405.430992:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575405.431031:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616499456/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575405.431057:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575405.431060:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575405.431061:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575405.431064:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7317 00000100:00080000:2.0:1788575405.431067:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575405.431070:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575405.431079:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575405.431092:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616499584/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575405.431103:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616499456:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788575405.431104:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575405.431112:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575405.431115:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575405.431119:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7317 00000100:00080000:2.0:1788575405.431123:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575405.431126:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575405.431131:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575405.431143:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616499712/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575405.431166:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575405.431193:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575405/real 1788575405] req@ffff9ada60626d80 x1875449616499456/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575430 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575405.431206:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616499456/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575430 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575405.431213:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575405.431216:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575405.431218:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616499456:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575405.431235:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616499584:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575405.431265:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575405.431273:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575405/real 1788575405] req@ffff9ada60625180 x1875449616499584/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575430 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575405.431281:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616499584/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575430 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575405.431289:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575405.431293:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575405.431296:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616499584:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575405.431301:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616499712:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575405.431319:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575405.431341:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575405/real 1788575405] req@ffff9ada60627480 x1875449616499712/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575430 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575405.431354:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616499712/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575430 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575405.431360:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575405.431362:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575405.431364:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616499712:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575405.622952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575405.943017:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575406.646981:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575406.967015:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575407.670950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575407.991485:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575408.694974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575409.015035:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575409.718985:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575410.039088:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575410.550912:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575410.550922:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575410.550926:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575410.550936:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7323 00000100:00080000:2.0:1788575410.550946:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575410.550953:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575410.551004:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616499840/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575410.551031:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575410.551035:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575410.551037:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575410.551041:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7323 00000100:00080000:2.0:1788575410.551045:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575410.551048:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575410.551058:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575410.551067:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616499840:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788575410.551071:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616499968/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575410.551080:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575410.551082:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575410.551084:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575410.551086:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7323 00000100:00080000:2.0:1788575410.551088:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575410.551091:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575410.551095:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575410.551102:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616500096/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575410.551147:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575410.551178:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575410/real 1788575410] req@ffff9ada60624e00 x1875449616499840/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575435 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575410.551193:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616499840/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575435 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575410.551211:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575410.551215:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575410.551218:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616499840:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575410.551247:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616499968:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575410.551265:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575410.551274:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575410/real 1788575410] req@ffff9ada60625f80 x1875449616499968/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575435 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575410.551281:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616499968/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575435 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575410.551287:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575410.551289:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575410.551291:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616499968:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575410.551294:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616500096:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575410.551304:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575410.551309:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575410/real 1788575410] req@ffff9ada60625c00 x1875449616500096/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575435 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575410.551316:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616500096/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575435 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575410.551321:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575410.551322:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575410.551324:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616500096:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575410.742974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575411.063013:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575411.766954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575412.086990:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575412.790957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575413.110993:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575413.814934:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575414.135023:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575414.838954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575415.159045:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575415.670939:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575415.670947:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575415.670949:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575415.670957:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7328 00000100:00080000:1.0:1788575415.670966:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575415.670971:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575415.671002:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e300 x1875449616500224/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575415.671028:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575415.671032:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575415.671033:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575415.671036:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7328 00000100:00080000:1.0:1788575415.671039:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575415.671042:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575415.671049:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575415.671059:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f480 x1875449616500352/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575415.671066:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575415.671068:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788575415.671069:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575415.671071:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7328 00000100:00080000:1.0:1788575415.671073:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575415.671075:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575415.671079:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575415.671081:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616500224:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788575415.671086:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ea00 x1875449616500480/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575415.671163:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575415.671195:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575415/real 1788575415] req@ffff9ada4566e300 x1875449616500224/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575440 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575415.671213:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e300 x1875449616500224/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575440 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575415.671223:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575415.671226:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575415.671230:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616500224:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575415.671237:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616500352:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575415.671256:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575415.671266:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575415/real 1788575415] req@ffff9ada4566f480 x1875449616500352/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575440 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575415.671278:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f480 x1875449616500352/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575440 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575415.671286:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575415.671289:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575415.671292:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616500352:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575415.671312:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616500480:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575415.671330:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575415.671340:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575415/real 1788575415] req@ffff9ada4566ea00 x1875449616500480/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575440 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575415.671358:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ea00 x1875449616500480/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575440 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575415.671366:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575415.671369:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575415.671372:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616500480:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575415.862945:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575416.183016:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575416.886983:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575417.206962:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575417.910972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575418.231093:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575418.934977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575419.255045:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575419.958944:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575420.279035:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575420.791002:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575420.791012:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575420.791016:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575420.791026:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7333 00000100:00080000:2.0:1788575420.791037:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575420.791045:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575420.791087:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616500608/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575420.791127:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575420.791130:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575420.791131:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575420.791135:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7333 00000100:00080000:2.0:1788575420.791138:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575420.791141:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575420.791150:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575420.791161:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616500736/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575420.791174:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575420.791178:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575420.791179:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575420.791182:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7333 00000100:00080000:2.0:1788575420.791186:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575420.791190:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575420.791194:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575420.791202:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616500864/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575420.791219:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616500608:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788575420.791283:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575420.791319:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575420/real 1788575420] req@ffff9ada60625880 x1875449616500608/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575445 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575420.791338:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616500608/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575445 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575420.791349:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575420.791353:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575420.791357:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616500608:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575420.791364:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616500736:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575420.791389:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575420.791399:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575420/real 1788575420] req@ffff9ada60624a80 x1875449616500736/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575445 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575420.791408:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616500736/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575445 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575420.791416:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575420.791420:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575420.791423:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616500736:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575420.791447:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616500864:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575420.791475:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575420.791486:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575420/real 1788575420] req@ffff9ada60625500 x1875449616500864/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575445 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575420.791494:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616500864/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575445 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575420.791499:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575420.791501:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575420.791503:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616500864:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575420.983002:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575421.303011:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575422.006973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575422.327053:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575423.030983:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575423.351346:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575424.054969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575424.374994:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575425.078995:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575425.399086:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575425.911005:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575425.911018:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575425.911024:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575425.911038:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7338 00000100:00080000:2.0:1788575425.911071:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575425.911082:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575425.911130:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616500992/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575425.911162:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575425.911166:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575425.911169:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575425.911173:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7338 00000100:00080000:2.0:1788575425.911178:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575425.911183:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575425.911195:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575425.911209:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616500992:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788575425.911211:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616501120/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575425.911225:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575425.911228:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575425.911230:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575425.911234:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7338 00000100:00080000:2.0:1788575425.911237:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575425.911240:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575425.911244:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575425.911254:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616501248/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575425.911294:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575425.911334:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575425/real 1788575425] req@ffff9ada60626a00 x1875449616500992/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575450 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575425.911351:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616500992/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575450 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575425.911360:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575425.911363:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575425.911366:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616500992:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575425.911387:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616501120:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575425.911405:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575425.911413:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575425/real 1788575425] req@ffff9ada60624380 x1875449616501120/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575450 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575425.911422:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616501120/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575450 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575425.911429:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575425.911432:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575425.911436:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616501120:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575425.911442:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616501248:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575425.911462:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575425.911474:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575425/real 1788575425] req@ffff9ada60626680 x1875449616501248/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575450 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575425.911488:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616501248/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575450 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575425.911494:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575425.911496:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575425.911499:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616501248:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575426.102978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575426.423093:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575427.126942:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575427.447081:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575428.150943:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575428.471021:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575429.174940:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575429.495035:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575430.198993:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575430.519145:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575431.030987:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575431.030995:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575431.030998:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575431.031020:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7343 00000100:00080000:0.0:1788575431.031028:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575431.031033:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575431.031062:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d180 x1875449616501376/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575431.031096:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575431.031100:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575431.031102:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575431.031106:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7343 00000100:00080000:0.0:1788575431.031110:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575431.031114:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575431.031123:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575431.031136:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616501504/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575431.031145:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1788575431.031145:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616501376:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788575431.031153:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575431.031155:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575431.031158:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7343 00000100:00080000:0.0:1788575431.031160:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575431.031163:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575431.031166:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575431.031175:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616501632/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575431.031200:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575431.031227:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575431/real 1788575431] req@ffff9ada4557d180 x1875449616501376/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575456 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575431.031239:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d180 x1875449616501376/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575456 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575431.031246:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575431.031248:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575431.031251:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616501376:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575431.031267:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616501504:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575431.031279:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575431.031286:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575431/real 1788575431] req@ffff9ada4557e680 x1875449616501504/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575456 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575431.031293:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616501504/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575456 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575431.031298:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575431.031300:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575431.031302:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616501504:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575431.031305:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616501632:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575431.031314:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575431.031318:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575431/real 1788575431] req@ffff9ada4557ed80 x1875449616501632/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575456 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575431.031324:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616501632/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575456 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575431.031329:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575431.031330:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575431.031332:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616501632:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575431.222920:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575431.543026:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575432.246940:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575432.567022:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575433.270954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575433.591012:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575434.294984:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575434.615053:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575435.319035:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575435.639054:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575436.151082:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575436.151095:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575436.151101:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575436.151113:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7348 00000100:00080000:0.0:1788575436.151127:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575436.151135:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575436.151187:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616501760/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575436.151224:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575436.151229:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575436.151231:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575436.151236:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7348 00000100:00080000:0.0:1788575436.151241:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575436.151245:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575436.151257:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575436.151271:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616501888/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575436.151283:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575436.151287:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575436.151290:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575436.151294:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7348 00000100:00080000:0.0:1788575436.151298:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575436.151303:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575436.151309:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575436.151319:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616501760:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575436.151331:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616502016/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575436.151400:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575436.151448:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575436/real 1788575436] req@ffff9ada4557fb80 x1875449616501760/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575461 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575436.151464:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616501760/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575461 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575436.151473:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575436.151476:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575436.151479:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616501760:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575436.151486:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616501888:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575436.151504:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575436.151513:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575436/real 1788575436] req@ffff9ada4557e300 x1875449616501888/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575461 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575436.151521:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616501888/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575461 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575436.151527:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575436.151529:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575436.151531:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616501888:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575436.151561:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616502016:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575436.151574:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575436.151581:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575436/real 1788575436] req@ffff9ada4557f800 x1875449616502016/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575461 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575436.151589:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616502016/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575461 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575436.151599:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575436.151601:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575436.151603:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616502016:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575436.343016:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575436.663011:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575437.366967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575437.687054:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575438.390949:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575438.711013:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575439.414993:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575439.735135:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575440.439019:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575440.759122:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575441.271065:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575441.271079:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575441.271085:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575441.271098:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7353 00000100:00080000:2.0:1788575441.271112:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575441.271121:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575441.271175:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627100 x1875449616502144/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575441.271202:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575441.271205:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575441.271207:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575441.271210:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7353 00000100:00080000:2.0:1788575441.271213:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575441.271215:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575441.271224:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575441.271236:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624700 x1875449616502272/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575441.271245:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575441.271248:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575441.271250:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575441.271252:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7353 00000100:00080000:2.0:1788575441.271254:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575441.271257:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575441.271260:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575441.271271:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616502400/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575441.271273:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616502144:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788575441.271358:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575441.271403:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575441/real 1788575441] req@ffff9ada60627100 x1875449616502144/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575466 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575441.271426:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627100 x1875449616502144/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575466 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575441.271438:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575441.271442:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575441.271446:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616502144:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575441.271454:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616502272:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575441.271481:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575441.271494:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575441/real 1788575441] req@ffff9ada60624700 x1875449616502272/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575466 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575441.271516:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624700 x1875449616502272/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575466 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575441.271526:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575441.271530:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575441.271534:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616502272:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575441.271564:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616502400:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575441.271591:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575441.271604:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575441/real 1788575441] req@ffff9ada60626680 x1875449616502400/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575466 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575441.271619:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616502400/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575466 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575441.271627:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575441.271632:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575441.271636:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616502400:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575441.462997:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575441.783050:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575442.486972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575442.807050:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575443.510952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575443.831060:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575444.534957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575444.855033:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575445.558979:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575445.879014:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788575446.390989:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575446.391005:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788575446.391011:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575446.391021:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7358 00000100:00080000:3.0:1788575446.391033:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575446.391041:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575446.391096:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0e00 x1875449616502528/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575446.391124:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575446.391127:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788575446.391129:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575446.391135:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7358 00000100:00080000:3.0:1788575446.391139:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575446.391143:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575446.391154:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575446.391163:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2a00 x1875449616502656/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575446.391171:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575446.391175:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788575446.391177:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1788575446.391183:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616502528:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788575446.391186:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7358 00000100:00080000:3.0:1788575446.391197:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575446.391204:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575446.391212:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575446.391227:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1500 x1875449616502784/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788575446.391300:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616502656:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575446.391325:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616502784:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575446.582997:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575446.903073:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575447.606970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575447.926977:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575448.630922:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575448.951052:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1788575449.463105:0:2371:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:1.0:1788575449.463150:0:2371:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1788575449.463165:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:1.0:1788575449.463200:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:1.0:1788575449.463214:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:0.0:1788575449.463246:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575446/real 1788575449] req@ffff9ada604e2a00 x1875449616502656/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575471 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575449.463270:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2a00 x1875449616502656/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575471 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575449.463283:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575449.463287:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575449.463290:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616502656:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575449.463295:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575446/real 1788575449] req@ffff9ada604e1500 x1875449616502784/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575471 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575449.463306:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1500 x1875449616502784/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575471 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575449.463314:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575449.463317:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575449.463320:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616502784:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575449.463327:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575446/real 1788575449] req@ffff9ada604e0e00 x1875449616502528/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575471 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575449.463338:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0e00 x1875449616502528/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575471 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575449.463346:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575449.463350:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575449.463353:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616502528:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575449.654960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575449.975010:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575450.678925:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575450.998985:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575451.510940:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575451.510947:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575451.510950:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575451.510958:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7364 00000100:00080000:0.0:1788575451.510967:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575451.510974:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575451.511006:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616502912/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575451.511029:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575451.511032:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575451.511033:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575451.511036:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7364 00000100:00080000:0.0:1788575451.511039:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575451.511041:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575451.511050:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575451.511062:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616503040/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575451.511068:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616502912:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788575451.511079:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575451.511083:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575451.511085:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575451.511089:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7364 00000100:00080000:0.0:1788575451.511094:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575451.511098:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575451.511103:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575451.511113:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616503168/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575451.511127:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575451.511159:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575451/real 1788575451] req@ffff9ada4557dc00 x1875449616502912/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575476 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575451.511178:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616502912/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575476 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575451.511187:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575451.511190:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575451.511192:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616502912:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575451.511230:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616503040:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575451.511253:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575451.511266:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575451/real 1788575451] req@ffff9ada4557d880 x1875449616503040/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575476 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575451.511278:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616503040/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575476 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575451.511285:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575451.511289:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575451.511294:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616503040:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575451.511299:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616503168:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575451.511320:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575451.511330:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575451/real 1788575451] req@ffff9ada4557ca80 x1875449616503168/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575476 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575451.511341:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616503168/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575476 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575451.511348:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575451.511352:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575451.511355:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616503168:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575451.702940:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575452.022998:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575452.726913:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575453.047022:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575453.750955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575454.071038:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575454.774944:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575455.095027:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575455.798964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575456.118984:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575456.630956:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575456.630963:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575456.630967:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575456.630975:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7369 00000100:00080000:2.0:1788575456.630984:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575456.630990:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575456.631022:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616503296/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575456.631047:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575456.631050:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575456.631052:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575456.631055:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7369 00000100:00080000:2.0:1788575456.631058:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575456.631061:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575456.631071:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575456.631082:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616503424/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575456.631093:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616503296:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788575456.631094:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575456.631101:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575456.631103:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575456.631107:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7369 00000100:00080000:2.0:1788575456.631111:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575456.631114:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575456.631119:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575456.631131:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616503552/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575456.631148:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575456.631175:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575456/real 1788575456] req@ffff9ada60624380 x1875449616503296/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575481 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575456.631187:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616503296/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575481 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575456.631198:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575456.631200:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575456.631203:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616503296:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575456.631218:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616503424:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575456.631230:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575456.631237:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575456/real 1788575456] req@ffff9ada60626a00 x1875449616503424/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575481 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575456.631243:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616503424/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575481 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575456.631248:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575456.631250:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575456.631252:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616503424:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575456.631255:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616503552:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575456.631263:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575456.631268:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575456/real 1788575456] req@ffff9ada60625500 x1875449616503552/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575481 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575456.631274:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616503552/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575481 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575456.631278:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575456.631280:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575456.631281:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616503552:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575456.822961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575457.142993:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575457.846952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575458.166995:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575458.870958:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575459.191042:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575459.894955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575460.215051:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575460.918962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575461.239008:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788575461.750979:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575461.750989:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788575461.750993:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575461.751002:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7374 00000100:00080000:3.0:1788575461.751011:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575461.751019:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575461.751054:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0e00 x1875449616503680/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575461.751075:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575461.751078:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788575461.751079:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575461.751081:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7374 00000100:00080000:3.0:1788575461.751084:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575461.751086:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575461.751094:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575461.751101:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1500 x1875449616503808/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575461.751115:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575461.751118:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788575461.751122:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1788575461.751123:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616503680:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788575461.751127:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7374 00000100:00080000:3.0:1788575461.751132:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575461.751134:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575461.751138:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575461.751147:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2a00 x1875449616503936/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575461.751178:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575461.751204:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575461/real 1788575461] req@ffff9ada604e0e00 x1875449616503680/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575486 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575461.751216:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0e00 x1875449616503680/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575486 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575461.751223:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575461.751225:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575461.751228:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616503680:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575461.751244:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616503808:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575461.751257:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575461.751264:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575461/real 1788575461] req@ffff9ada604e1500 x1875449616503808/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575486 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575461.751271:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1500 x1875449616503808/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575486 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575461.751280:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575461.751282:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575461.751284:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616503808:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575461.751287:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616503936:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575461.751295:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575461.751300:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575461/real 1788575461] req@ffff9ada604e2a00 x1875449616503936/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575486 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575461.751306:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2a00 x1875449616503936/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575486 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575461.751311:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575461.751312:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575461.751314:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616503936:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575461.942959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575462.263024:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575462.966936:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575463.286974:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575463.990968:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575464.311046:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575465.014987:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575465.334994:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575466.038962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575466.359013:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575466.871016:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575466.871026:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575466.871030:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575466.871039:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7379 00000100:00080000:0.0:1788575466.871050:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575466.871058:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575466.871094:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616504064/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575466.871125:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575466.871128:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575466.871130:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575466.871135:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7379 00000100:00080000:0.0:1788575466.871139:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575466.871143:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575466.871153:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575466.871155:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616504064:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575466.871168:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ce00 x1875449616504192/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575466.871179:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575466.871182:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575466.871184:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575466.871187:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7379 00000100:00080000:0.0:1788575466.871191:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575466.871195:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575466.871200:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575466.871212:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616504320/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575466.871215:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575466.871251:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575466/real 1788575466] req@ffff9ada4557c700 x1875449616504064/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575491 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575466.871270:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616504064/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575491 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575466.871277:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575466.871280:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575466.871282:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616504064:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575466.871299:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616504192:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575466.871315:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575466.871323:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575466/real 1788575466] req@ffff9ada4557ce00 x1875449616504192/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575491 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575466.871331:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ce00 x1875449616504192/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575491 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575466.871336:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575466.871338:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575466.871340:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616504192:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575466.871343:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616504320:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575466.871352:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575466.871357:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575466/real 1788575466] req@ffff9ada4557f480 x1875449616504320/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575491 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575466.871364:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616504320/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575491 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575466.871368:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575466.871370:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575466.871372:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616504320:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575467.062955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575467.383012:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575468.086942:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575468.407074:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575469.110945:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575469.431025:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575470.134929:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575470.455054:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575471.159022:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575471.479027:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788575471.990969:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575471.990989:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788575471.990994:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575471.991008:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7384 00000100:00080000:3.0:1788575471.991022:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575471.991031:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575471.991078:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3100 x1875449616504448/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575471.991109:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575471.991113:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788575471.991116:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575471.991121:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7384 00000100:00080000:3.0:1788575471.991125:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575471.991129:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575471.991151:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575471.991157:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616504448:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575471.991167:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2680 x1875449616504576/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575471.991180:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575471.991184:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788575471.991187:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575471.991191:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7384 00000100:00080000:3.0:1788575471.991195:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575471.991200:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575471.991205:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575471.991216:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1f80 x1875449616504704/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575471.991236:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575471.991276:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575471/real 1788575471] req@ffff9ada604e3100 x1875449616504448/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575496 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575471.991293:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3100 x1875449616504448/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575496 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575471.991307:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575471.991310:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575471.991313:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616504448:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575471.991337:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616504576:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575471.991353:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575471.991362:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575471/real 1788575471] req@ffff9ada604e2680 x1875449616504576/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575496 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575471.991371:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2680 x1875449616504576/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575496 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575471.991377:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575471.991379:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575471.991381:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616504576:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575471.991385:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616504704:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575471.991398:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575471.991404:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575471/real 1788575471] req@ffff9ada604e1f80 x1875449616504704/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575496 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575471.991411:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1f80 x1875449616504704/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575496 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575471.991421:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575471.991423:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575471.991425:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616504704:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575472.182972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575472.503050:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575473.207010:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575473.527076:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575474.230949:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575474.551051:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575475.254950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575475.575058:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575476.278975:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575476.599036:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575477.110978:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575477.110986:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575477.110990:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575477.110996:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7389 00000100:00080000:1.0:1788575477.111005:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575477.111010:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575477.111039:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d500 x1875449616504832/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575477.111061:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575477.111063:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575477.111065:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575477.111067:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7389 00000100:00080000:1.0:1788575477.111069:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575477.111070:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575477.111076:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575477.111083:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ca80 x1875449616504960/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575477.111090:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575477.111092:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788575477.111093:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575477.111095:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7389 00000100:00080000:1.0:1788575477.111097:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575477.111099:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575477.111102:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575477.111109:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d180 x1875449616505088/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788575477.111111:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616504832:192.168.202.113@tcp:250:kworker.0 00000800:00000100:0.0:1788575477.111173:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575477.111203:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575477/real 1788575477] req@ffff9ada4566d500 x1875449616504832/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575502 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575477.111219:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d500 x1875449616504832/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575502 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575477.111230:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575477.111233:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575477.111237:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616504832:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575477.111244:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616504960:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575477.111263:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575477.111273:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575477/real 1788575477] req@ffff9ada4566ca80 x1875449616504960/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575502 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575477.111297:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ca80 x1875449616504960/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575502 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575477.111305:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575477.111309:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575477.111312:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616504960:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575477.111339:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616505088:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575477.111356:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575477.111365:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575477/real 1788575477] req@ffff9ada4566d180 x1875449616505088/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575502 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575477.111375:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d180 x1875449616505088/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575502 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575477.111383:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575477.111386:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575477.111389:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616505088:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575477.302971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575477.622969:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575478.326933:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575478.646999:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575479.350929:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575479.670967:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575480.374979:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575480.695040:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575481.398963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575481.719051:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788575482.230989:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575482.230999:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788575482.231003:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575482.231014:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7394 00000100:00080000:3.0:1788575482.231023:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575482.231029:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575482.231064:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3b80 x1875449616505216/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575482.231088:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575482.231092:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788575482.231094:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575482.231097:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7394 00000100:00080000:3.0:1788575482.231100:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575482.231105:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575482.231112:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575482.231123:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1c00 x1875449616505344/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575482.231135:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575482.231139:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788575482.231141:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575482.231145:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7394 00000100:00080000:3.0:1788575482.231149:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575482.231153:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575482.231160:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575482.231176:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0000 x1875449616505472/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788575482.231179:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616505216:192.168.202.113@tcp:250:kworker.0 00000800:00000100:0.0:1788575482.231283:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575482.231341:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575482/real 1788575482] req@ffff9ada604e3b80 x1875449616505216/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575507 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575482.231363:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3b80 x1875449616505216/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575507 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575482.231376:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575482.231381:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575482.231386:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616505216:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575482.231395:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616505344:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575482.231421:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575482.231463:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575482/real 1788575482] req@ffff9ada604e1c00 x1875449616505344/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575507 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575482.231477:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1c00 x1875449616505344/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575507 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575482.231486:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575482.231489:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575482.231492:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616505344:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575482.231522:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616505472:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575482.231545:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575482.231556:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575482/real 1788575482] req@ffff9ada604e0000 x1875449616505472/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575507 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575482.231568:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0000 x1875449616505472/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575507 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575482.231575:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575482.231578:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575482.231582:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616505472:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575482.422965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575482.743029:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575483.446985:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575483.767035:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575484.470934:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575484.790989:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575485.494960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575485.815734:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575486.518951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575486.839017:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788575487.350983:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575487.350996:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788575487.351001:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575487.351013:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7399 00000100:00080000:3.0:1788575487.351028:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575487.351037:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575487.351116:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2300 x1875449616505600/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575487.351148:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575487.351152:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788575487.351154:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575487.351158:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7399 00000100:00080000:3.0:1788575487.351161:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575487.351164:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575487.351174:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575487.351194:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0a80 x1875449616505728/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788575487.351206:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616505600:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788575487.351208:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575487.351216:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788575487.351219:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575487.351223:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7399 00000100:00080000:3.0:1788575487.351227:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575487.351232:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575487.351237:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575487.351265:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2d80 x1875449616505856/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575487.351286:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575487.351329:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575487/real 1788575487] req@ffff9ada604e2300 x1875449616505600/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575512 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575487.351353:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2300 x1875449616505600/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575512 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575487.351364:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575487.351368:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575487.351372:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616505600:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575487.351397:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616505728:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575487.351418:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575487.351429:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575487/real 1788575487] req@ffff9ada604e0a80 x1875449616505728/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575512 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575487.351441:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0a80 x1875449616505728/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575512 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575487.351449:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575487.351452:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575487.351455:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616505728:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575487.351461:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616505856:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575487.351479:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575487.351487:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575487/real 1788575487] req@ffff9ada604e2d80 x1875449616505856/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575512 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575487.351499:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2d80 x1875449616505856/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575512 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575487.351512:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575487.351516:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575487.351519:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616505856:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575487.542961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575487.863074:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575488.566964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575488.886978:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575489.590998:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575489.911020:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575490.614966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575490.935014:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575491.638969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575491.959017:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788575492.470991:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575492.471004:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788575492.471009:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575492.471021:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7405 00000100:00080000:3.0:1788575492.471035:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575492.471043:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575492.471089:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1880 x1875449616505984/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575492.471124:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575492.471128:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788575492.471130:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575492.471133:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7405 00000100:00080000:3.0:1788575492.471137:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575492.471141:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575492.471152:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575492.471166:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3800 x1875449616506112/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788575492.471176:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616505984:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788575492.471180:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575492.471184:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788575492.471185:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575492.471200:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7405 00000100:00080000:3.0:1788575492.471207:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575492.471212:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575492.471217:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575492.471230:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1180 x1875449616506240/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575492.471254:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575492.471300:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575492/real 1788575492] req@ffff9ada604e1880 x1875449616505984/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575517 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575492.471319:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1880 x1875449616505984/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575517 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575492.471327:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575492.471330:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575492.471334:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616505984:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575492.471362:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616506112:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575492.471386:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575492.471396:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575492/real 1788575492] req@ffff9ada604e3800 x1875449616506112/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575517 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575492.471407:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3800 x1875449616506112/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575517 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575492.471415:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575492.471419:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575492.471422:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616506112:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575492.471428:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616506240:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575492.471449:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575492.471457:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575492/real 1788575492] req@ffff9ada604e1180 x1875449616506240/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575517 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575492.471475:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1180 x1875449616506240/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575517 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575492.471483:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575492.471486:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575492.471489:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616506240:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575492.662999:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575492.983010:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575493.686957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575494.007044:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575494.710913:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575495.031065:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575495.734969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575496.055048:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575496.758946:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575497.078994:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575497.590993:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575497.591000:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575497.591003:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575497.591011:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7410 00000100:00080000:1.0:1788575497.591019:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575497.591024:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575497.591052:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f100 x1875449616506368/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575497.591072:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575497.591074:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575497.591076:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575497.591078:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7410 00000100:00080000:1.0:1788575497.591081:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575497.591083:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575497.591090:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575497.591098:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c380 x1875449616506496/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575497.591104:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575497.591107:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788575497.591108:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575497.591110:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7410 00000100:00080000:1.0:1788575497.591113:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1788575497.591115:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616506368:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788575497.591117:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575497.591124:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575497.591131:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c700 x1875449616506624/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575497.591169:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575497.591195:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575497/real 1788575497] req@ffff9ada4566f100 x1875449616506368/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575522 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575497.591206:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f100 x1875449616506368/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575522 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575497.591213:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575497.591216:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575497.591219:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616506368:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575497.591223:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616506496:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575497.591235:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575497.591242:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575497/real 1788575497] req@ffff9ada4566c380 x1875449616506496/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575522 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575497.591248:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c380 x1875449616506496/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575522 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575497.591253:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575497.591255:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575497.591257:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616506496:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575497.591272:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616506624:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575497.591281:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575497.591287:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575497/real 1788575497] req@ffff9ada4566c700 x1875449616506624/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575522 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575497.591293:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c700 x1875449616506624/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575522 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575497.591297:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575497.591299:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575497.591300:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616506624:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575497.782959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575498.103080:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575498.806970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575499.126975:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575499.830951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575500.151069:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575500.854980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575501.175020:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575501.878955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575502.198991:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575502.710949:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575502.710956:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575502.710960:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575502.710967:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7415 00000100:00080000:2.0:1788575502.710975:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575502.710980:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575502.711010:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616506752/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575502.711044:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575502.711046:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575502.711047:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575502.711050:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7415 00000100:00080000:2.0:1788575502.711053:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575502.711056:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575502.711062:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575502.711071:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616506880/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575502.711084:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575502.711090:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575502.711091:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575502.711096:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7415 00000100:00080000:2.0:1788575502.711099:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575502.711103:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575502.711108:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616506752:192.168.202.113@tcp:250:kworker.0 00000008:00100000:2.0:1788575502.711110:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575502.711125:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616507008/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575502.711162:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575502.711194:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575502/real 1788575502] req@ffff9ada60624a80 x1875449616506752/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575527 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575502.711213:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616506752/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575527 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575502.711224:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575502.711227:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575502.711231:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616506752:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575502.711238:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616506880:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575502.711255:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575502.711263:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575502/real 1788575502] req@ffff9ada60625880 x1875449616506880/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575527 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575502.711271:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616506880/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575527 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575502.711276:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575502.711278:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575502.711279:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616506880:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575502.711296:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616507008:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575502.711306:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575502.711312:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575502/real 1788575502] req@ffff9ada60625c00 x1875449616507008/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575527 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575502.711319:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616507008/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575527 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575502.711325:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575502.711327:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575502.711330:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616507008:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575502.902957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575503.223055:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575503.926979:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575504.247021:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575504.950947:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575505.270973:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575505.974975:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575506.294971:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575506.998973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575507.319035:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575507.830992:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575507.831000:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575507.831004:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575507.831011:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7420 00000100:00080000:1.0:1788575507.831019:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575507.831024:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575507.831052:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ce00 x1875449616507136/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575507.831072:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575507.831075:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575507.831076:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575507.831080:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7420 00000100:00080000:1.0:1788575507.831082:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575507.831086:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575507.831093:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575507.831107:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d880 x1875449616507264/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575507.831118:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575507.831121:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788575507.831122:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575507.831127:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7420 00000100:00100000:0.0:1788575507.831132:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616507136:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788575507.831132:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575507.831141:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575507.831146:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575507.831156:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e680 x1875449616507392/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575507.831217:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575507.831250:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575507/real 1788575507] req@ffff9ada4566ce00 x1875449616507136/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575532 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575507.831268:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ce00 x1875449616507136/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575532 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575507.831278:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575507.831282:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575507.831285:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616507136:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575507.831292:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616507264:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575507.831314:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575507.831328:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575507/real 1788575507] req@ffff9ada4566d880 x1875449616507264/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575532 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575507.831345:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d880 x1875449616507264/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575532 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575507.831354:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575507.831357:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575507.831361:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616507264:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575507.831381:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616507392:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575507.831401:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575507.831412:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575507/real 1788575507] req@ffff9ada4566e680 x1875449616507392/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575532 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575507.831424:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e680 x1875449616507392/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575532 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575507.831432:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575507.831436:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575507.831439:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616507392:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575508.022974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575508.342986:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575509.046939:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575509.367195:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575510.070992:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575510.391048:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575511.094924:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575511.415051:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575512.118954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575512.439013:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575512.950969:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575512.950976:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575512.950979:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575512.950988:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7425 00000100:00080000:2.0:1788575512.950997:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575512.951003:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575512.951045:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616507520/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575512.951072:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575512.951076:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575512.951077:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575512.951081:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7425 00000100:00080000:2.0:1788575512.951085:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575512.951089:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575512.951098:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575512.951108:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616507648/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575512.951116:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575512.951120:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:0.0:1788575512.951121:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616507520:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788575512.951121:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575512.951128:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7425 00000100:00080000:2.0:1788575512.951131:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575512.951133:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575512.951136:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575512.951144:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616507776/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788575512.951204:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616507648:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575512.951227:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616507776:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575513.142957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575513.463019:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575514.166953:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575514.487018:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575515.190955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575515.511017:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1788575516.023166:0:2370:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:2.0:1788575516.023206:0:2370:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1788575516.023222:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788575516.023269:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788575516.023278:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:0.0:1788575516.023379:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575512/real 1788575516] req@ffff9ada60624e00 x1875449616507648/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575537 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575516.023417:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616507648/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575537 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575516.023430:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575516.023435:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575516.023440:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616507648:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575516.023448:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575512/real 1788575516] req@ffff9ada60627480 x1875449616507776/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575537 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575516.023458:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616507776/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575537 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575516.023467:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575516.023471:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575516.023474:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616507776:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575516.023481:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575512/real 1788575516] req@ffff9ada60625f80 x1875449616507520/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575537 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575516.023492:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616507520/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575537 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575516.023498:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575516.023502:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575516.023506:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616507520:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575516.214959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575516.535052:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575517.238953:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575517.559033:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575518.071094:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575518.071109:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575518.071120:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575518.071133:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7430 00000100:00080000:2.0:1788575518.071148:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575518.071156:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575518.071201:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616507904/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575518.071236:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575518.071240:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575518.071242:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575518.071246:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7430 00000100:00080000:2.0:1788575518.071254:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575518.071259:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575518.071271:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575518.071285:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616508032/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575518.071295:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575518.071299:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575518.071301:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575518.071304:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7430 00000100:00080000:2.0:1788575518.071308:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575518.071312:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575518.071317:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575518.071328:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616508160/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788575518.071439:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616507904:192.168.202.113@tcp:250:kworker.0 00000800:00000100:0.0:1788575518.071555:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575518.071599:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575518/real 1788575518] req@ffff9ada60625180 x1875449616507904/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575543 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575518.071618:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616507904/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575543 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575518.071630:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575518.071635:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575518.071639:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616507904:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575518.071646:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616508032:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575518.071669:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575518.071681:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575518/real 1788575518] req@ffff9ada60626d80 x1875449616508032/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575543 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575518.071693:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616508032/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575543 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575518.071701:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575518.071705:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575518.071707:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616508032:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575518.071713:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616508160:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575518.071733:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575518.071745:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575518/real 1788575518] req@ffff9ada60627b80 x1875449616508160/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575543 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575518.071757:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616508160/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575543 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575518.071764:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575518.071767:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575518.071770:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616508160:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575518.262985:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575518.583044:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575519.286930:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575519.607021:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575520.310969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575520.630979:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575521.334925:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575521.655015:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575522.358952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575522.679077:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575523.190986:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575523.190993:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575523.190996:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575523.191004:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7435 00000100:00080000:1.0:1788575523.191011:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575523.191017:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575523.191046:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e680 x1875449616508288/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575523.191066:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575523.191069:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575523.191070:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575523.191072:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7435 00000100:00080000:1.0:1788575523.191074:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575523.191076:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575523.191083:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575523.191090:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d880 x1875449616508416/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575523.191096:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575523.191099:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788575523.191100:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575523.191102:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7435 00000100:00080000:1.0:1788575523.191104:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575523.191106:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575523.191108:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575523.191115:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616508288:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788575523.191116:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ce00 x1875449616508544/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575523.191175:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575523.191204:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575523/real 1788575523] req@ffff9ada4566e680 x1875449616508288/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575548 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575523.191220:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e680 x1875449616508288/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575548 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575523.191226:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575523.191229:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575523.191231:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616508288:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575523.191236:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616508416:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575523.191249:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575523.191256:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575523/real 1788575523] req@ffff9ada4566d880 x1875449616508416/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575548 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575523.191267:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d880 x1875449616508416/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575548 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575523.191273:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575523.191275:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575523.191278:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616508416:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575523.191298:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616508544:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575523.191317:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575523.191329:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575523/real 1788575523] req@ffff9ada4566ce00 x1875449616508544/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575548 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575523.191341:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ce00 x1875449616508544/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575548 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575523.191348:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575523.191350:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575523.191353:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616508544:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575523.382926:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575523.702987:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575524.406949:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575524.727012:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575525.430959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575525.750996:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575526.454928:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575526.775061:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575527.478984:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575527.798991:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575528.310974:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575528.310982:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575528.310985:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575528.310993:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7440 00000100:00080000:0.0:1788575528.311000:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575528.311006:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575528.311034:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c000 x1875449616508672/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575528.311057:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575528.311059:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575528.311061:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575528.311063:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7440 00000100:00080000:0.0:1788575528.311065:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575528.311067:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575528.311074:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575528.311080:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c380 x1875449616508800/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575528.311088:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575528.311091:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575528.311092:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575528.311095:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7440 00000100:00080000:0.0:1788575528.311097:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575528.311099:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575528.311102:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575528.311112:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616508928/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575528.311112:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616508672:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788575528.311175:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575528.311207:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575528/real 1788575528] req@ffff9ada4557c000 x1875449616508672/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575553 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575528.311224:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c000 x1875449616508672/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575553 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575528.311233:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575528.311237:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575528.311240:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616508672:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575528.311245:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616508800:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575528.311266:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575528.311278:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575528/real 1788575528] req@ffff9ada4557c380 x1875449616508800/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575553 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575528.311289:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c380 x1875449616508800/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575553 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575528.311296:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575528.311299:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575528.311302:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616508800:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575528.311323:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616508928:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575528.311341:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575528.311351:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575528/real 1788575528] req@ffff9ada4557f480 x1875449616508928/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575553 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575528.311361:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616508928/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575553 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575528.311368:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575528.311370:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575528.311373:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616508928:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575528.502928:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575528.823053:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575529.526952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575529.846973:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575530.550928:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575530.871031:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575531.574977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575531.895023:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575532.599003:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575532.919066:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575533.430988:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575533.430997:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575533.431000:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575533.431008:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7445 00000100:00080000:1.0:1788575533.431017:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575533.431022:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575533.431057:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c700 x1875449616509056/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575533.431075:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575533.431077:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575533.431078:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575533.431080:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7445 00000100:00080000:1.0:1788575533.431082:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575533.431084:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575533.431091:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575533.431098:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c380 x1875449616509184/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575533.431106:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575533.431109:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788575533.431111:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575533.431115:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7445 00000100:00080000:1.0:1788575533.431118:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575533.431124:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575533.431126:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616509056:192.168.202.113@tcp:250:kworker.0 00000008:00100000:1.0:1788575533.431129:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575533.431142:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f100 x1875449616509312/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575533.431187:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575533.431219:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575533/real 1788575533] req@ffff9ada4566c700 x1875449616509056/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575558 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575533.431236:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c700 x1875449616509056/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575558 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575533.431246:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575533.431250:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575533.431253:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616509056:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575533.431260:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616509184:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575533.431280:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575533.431291:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575533/real 1788575533] req@ffff9ada4566c380 x1875449616509184/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575558 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575533.431303:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c380 x1875449616509184/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575558 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575533.431311:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575533.431315:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575533.431318:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616509184:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575533.431339:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616509312:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575533.431360:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575533.431370:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575533/real 1788575533] req@ffff9ada4566f100 x1875449616509312/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575558 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575533.431381:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f100 x1875449616509312/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575558 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575533.431388:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575533.431391:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575533.431394:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616509312:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575533.622988:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575533.943028:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575534.646963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575534.967082:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575535.670976:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575535.991002:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575536.694952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575537.015056:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575537.718977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575538.038968:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575538.550987:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575538.550998:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575538.551002:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575538.551012:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7451 00000100:00080000:2.0:1788575538.551024:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575538.551032:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575538.551076:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627800 x1875449616509440/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575538.551108:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575538.551112:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575538.551114:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575538.551118:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7451 00000100:00080000:2.0:1788575538.551121:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575538.551126:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575538.551143:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575538.551159:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626300 x1875449616509568/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575538.551160:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616509440:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788575538.551176:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575538.551179:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575538.551181:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575538.551185:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7451 00000100:00080000:2.0:1788575538.551189:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575538.551192:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575538.551197:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575538.551210:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616509696/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575538.551235:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575538.551273:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575538/real 1788575538] req@ffff9ada60627800 x1875449616509440/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575563 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575538.551293:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627800 x1875449616509440/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575563 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575538.551304:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575538.551308:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575538.551312:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616509440:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575538.551339:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616509568:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575538.551360:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575538.551369:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575538/real 1788575538] req@ffff9ada60626300 x1875449616509568/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575563 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575538.551380:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626300 x1875449616509568/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575563 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575538.551388:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575538.551392:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575538.551395:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616509568:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575538.551402:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616509696:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575538.551421:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575538.551431:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575538/real 1788575538] req@ffff9ada60627b80 x1875449616509696/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575563 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575538.551438:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616509696/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575563 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575538.551443:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575538.551445:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575538.551447:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616509696:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575538.742960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575539.063114:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575539.766994:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575540.087099:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575540.790944:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575541.111031:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575541.814970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575542.134989:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575542.838945:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575543.159011:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575543.670972:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575543.670982:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575543.670986:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575543.670994:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7456 00000100:00080000:2.0:1788575543.671001:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575543.671007:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575543.671036:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616509824/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575543.671058:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575543.671060:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575543.671061:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575543.671064:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7456 00000100:00080000:2.0:1788575543.671066:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575543.671067:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575543.671074:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575543.671081:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616509952/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575543.671099:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575543.671102:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575543.671104:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575543.671108:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7456 00000100:00080000:2.0:1788575543.671112:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1788575543.671117:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616509824:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788575543.671119:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575543.671126:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575543.671139:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616510080/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575543.671177:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575543.671202:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575543/real 1788575543] req@ffff9ada60626d80 x1875449616509824/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575568 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575543.671215:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616509824/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575568 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575543.671222:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575543.671226:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575543.671230:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616509824:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575543.671236:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616509952:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575543.671257:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575543.671268:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575543/real 1788575543] req@ffff9ada60625180 x1875449616509952/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575568 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575543.671280:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616509952/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575568 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575543.671288:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575543.671291:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575543.671294:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616509952:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575543.671313:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616510080:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575543.671326:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575543.671333:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575543/real 1788575543] req@ffff9ada60625f80 x1875449616510080/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575568 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575543.671344:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616510080/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575568 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575543.671351:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575543.671353:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575543.671357:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616510080:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575543.862969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575544.183035:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575544.886981:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788575545.207035:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575545.910937:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575546.231075:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575546.934951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575547.254967:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575547.958958:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788575548.278978:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788575548.790952:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575548.790959:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788575548.790963:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575548.790972:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7461 00000100:00080000:3.0:1788575548.790982:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575548.790990:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575548.791050:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0700 x1875449616510208/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575548.791076:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575548.791079:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788575548.791081:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575548.791085:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7461 00000100:00080000:3.0:1788575548.791089:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575548.791093:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575548.791102:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575548.791115:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3480 x1875449616510336/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788575548.791119:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616510208:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788575548.791132:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575548.791136:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788575548.791137:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575548.791142:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7461 00000100:00080000:3.0:1788575548.791153:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575548.791158:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575548.791163:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575548.791175:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0380 x1875449616510464/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575548.791186:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575548.791233:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575548/real 1788575548] req@ffff9ada604e0700 x1875449616510208/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575573 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575548.791250:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0700 x1875449616510208/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575573 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575548.791260:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575548.791263:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575548.791267:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616510208:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575548.791292:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616510336:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575548.791314:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575548.791326:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575548/real 1788575548] req@ffff9ada604e3480 x1875449616510336/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575573 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575548.791344:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3480 x1875449616510336/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575573 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575548.791352:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575548.791355:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575548.791358:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616510336:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575548.791363:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616510464:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575548.791380:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575548.791389:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575548/real 1788575548] req@ffff9ada604e0380 x1875449616510464/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575573 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575548.791400:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0380 x1875449616510464/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575573 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575548.791408:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575548.791411:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575548.791413:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616510464:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575548.982976:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575549.303052:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575550.006965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575550.326998:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575551.030973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788575551.351000:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575552.054954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575552.375016:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575553.078966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575553.399050:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575553.910990:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575553.911000:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575553.911005:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575553.911014:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7466 00000100:00080000:1.0:1788575553.911024:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575553.911031:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575553.911066:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d180 x1875449616510592/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575553.911123:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575553.911127:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575553.911129:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575553.911133:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7466 00000100:00080000:1.0:1788575553.911136:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575553.911140:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575553.911148:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575553.911160:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ca80 x1875449616510720/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575553.911169:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575553.911172:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788575553.911178:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575553.911182:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7466 00000100:00080000:1.0:1788575553.911185:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575553.911189:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575553.911194:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575553.911204:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d500 x1875449616510848/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788575553.911223:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616510592:192.168.202.113@tcp:250:kworker.0 00000800:00000100:0.0:1788575553.911284:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575553.911316:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575553/real 1788575553] req@ffff9ada4566d180 x1875449616510592/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575578 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575553.911338:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d180 x1875449616510592/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575578 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575553.911347:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575553.911351:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575553.911354:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616510592:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575553.911358:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616510720:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575553.911381:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575553.911389:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575553/real 1788575553] req@ffff9ada4566ca80 x1875449616510720/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575578 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575553.911396:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ca80 x1875449616510720/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575578 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575553.911401:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575553.911403:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575553.911404:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616510720:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575553.911420:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616510848:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575553.911429:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575553.911435:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575553/real 1788575553] req@ffff9ada4566d500 x1875449616510848/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575578 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575553.911441:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d500 x1875449616510848/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575578 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575553.911449:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575553.911451:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575553.911453:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616510848:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575554.102996:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788575554.423006:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575555.126945:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575555.447034:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575556.150956:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575556.471032:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575557.174962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788575557.494948:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575558.198984:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575558.519046:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575559.030969:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575559.030977:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575559.030980:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575559.030987:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7471 00000100:00080000:1.0:1788575559.031004:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575559.031011:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575559.031046:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ea00 x1875449616510976/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575559.031073:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575559.031077:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575559.031079:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575559.031083:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7471 00000100:00080000:1.0:1788575559.031087:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575559.031091:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575559.031107:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575559.031122:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f480 x1875449616511104/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788575559.031135:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616510976:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788575559.031145:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575559.031148:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788575559.031149:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575559.031152:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7471 00000100:00080000:1.0:1788575559.031155:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575559.031167:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575559.031171:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575559.031181:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e300 x1875449616511232/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575559.031210:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575559.031241:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575559/real 1788575559] req@ffff9ada4566ea00 x1875449616510976/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575584 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575559.031253:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ea00 x1875449616510976/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575584 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575559.031260:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575559.031263:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575559.031265:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616510976:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575559.031281:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616511104:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575559.031305:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575559.031314:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575559/real 1788575559] req@ffff9ada4566f480 x1875449616511104/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575584 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575559.031321:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f480 x1875449616511104/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575584 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575559.031325:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575559.031328:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575559.031331:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616511104:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575559.031337:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616511232:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575559.031354:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575559.031363:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575559/real 1788575559] req@ffff9ada4566e300 x1875449616511232/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575584 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575559.031375:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e300 x1875449616511232/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575584 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575559.031380:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575559.031382:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575559.031384:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616511232:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575559.222967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575559.543254:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575560.246931:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788575560.566975:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575561.270952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575561.591004:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575562.294921:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575562.615039:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575563.318979:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788575563.639040:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575564.151011:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575564.151024:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575564.151031:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575564.151043:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7476 00000100:00080000:2.0:1788575564.151059:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575564.151068:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575564.151117:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616511360/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575564.151153:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575564.151157:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575564.151160:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575564.151164:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7476 00000100:00080000:2.0:1788575564.151168:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575564.151172:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575564.151184:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575564.151199:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616511488/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575564.151212:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1788575564.151218:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616511360:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788575564.151218:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575564.151222:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575564.151231:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7476 00000100:00080000:2.0:1788575564.151235:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575564.151256:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575564.151262:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575564.151275:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616511616/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575564.151322:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575564.151360:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575564/real 1788575564] req@ffff9ada60627480 x1875449616511360/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575589 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575564.151379:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616511360/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575589 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575564.151390:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575564.151394:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575564.151397:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616511360:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575564.151423:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616511488:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575564.151444:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575564.151456:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575564/real 1788575564] req@ffff9ada60624e00 x1875449616511488/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575589 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575564.151466:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616511488/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575589 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575564.151474:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575564.151477:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575564.151480:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616511488:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575564.151485:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616511616:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575564.151502:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575564.151511:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575564/real 1788575564] req@ffff9ada60625c00 x1875449616511616/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575589 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575564.151521:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616511616/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575589 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575564.151528:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575564.151531:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575564.151534:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616511616:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575564.342982:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575564.662979:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575565.366951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575565.687086:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575566.390955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788575566.710970:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575567.414928:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575567.735072:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575568.438972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575568.759101:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575569.270990:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575569.271008:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575569.271013:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575569.271024:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7481 00000100:00080000:2.0:1788575569.271036:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575569.271044:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575569.271087:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616511744/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575569.271109:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575569.271112:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575569.271114:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575569.271117:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7481 00000100:00080000:2.0:1788575569.271121:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575569.271125:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575569.271136:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575569.271149:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616511872/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575569.271161:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575569.271165:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575569.271168:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1788575569.271169:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616511744:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788575569.271174:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7481 00000100:00080000:2.0:1788575569.271180:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575569.271186:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575569.271191:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575569.271203:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616512000/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575569.271260:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575569.271305:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575569/real 1788575569] req@ffff9ada60625880 x1875449616511744/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575594 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575569.271341:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616511744/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575594 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575569.271354:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575569.271359:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575569.271363:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616511744:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575569.271389:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616511872:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575569.271415:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575569.271428:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575569/real 1788575569] req@ffff9ada60624a80 x1875449616511872/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575594 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575569.271442:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616511872/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575594 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575569.271450:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575569.271454:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575569.271457:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616511872:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575569.271463:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616512000:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575569.271484:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575569.271495:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575569/real 1788575569] req@ffff9ada60625500 x1875449616512000/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575594 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575569.271514:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616512000/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575594 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575569.271522:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575569.271526:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575569.271529:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616512000:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575569.462971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788575569.783064:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575570.487422:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575570.807031:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575571.511013:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575571.831056:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575572.534974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788575572.854992:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575573.558971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575573.879281:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575574.390949:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575574.390962:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575574.390968:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575574.390980:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7486 00000100:00080000:2.0:1788575574.390994:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575574.391003:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575574.391053:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616512128/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575574.391085:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575574.391089:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575574.391091:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575574.391095:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7486 00000100:00080000:2.0:1788575574.391099:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575574.391104:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575574.391116:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575574.391132:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616512128:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788575574.391132:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616512256/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575574.391149:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575574.391153:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575574.391155:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575574.391159:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7486 00000100:00080000:2.0:1788575574.391163:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575574.391167:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575574.391172:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575574.391188:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616512384/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575574.391218:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575574.391263:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575574/real 1788575574] req@ffff9ada60626a00 x1875449616512128/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575599 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575574.391283:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616512128/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575599 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575574.391294:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575574.391299:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575574.391303:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616512128:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575574.391327:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616512256:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575574.391348:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575574.391359:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575574/real 1788575574] req@ffff9ada60624380 x1875449616512256/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575599 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575574.391376:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616512256/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575599 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575574.391383:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575574.391385:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575574.391389:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616512256:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575574.391394:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616512384:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575574.391440:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575574.391450:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575574/real 1788575574] req@ffff9ada60626680 x1875449616512384/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575599 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575574.391461:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616512384/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575599 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575574.391467:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575574.391470:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575574.391473:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616512384:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575574.582991:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575574.903015:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575575.606990:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788575575.926980:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575576.630940:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575576.950967:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575577.654965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575577.975018:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575578.678968:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575578.999018:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575579.511004:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575579.511014:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575579.511018:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575579.511029:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7492 00000100:00080000:0.0:1788575579.511040:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575579.511050:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575579.511101:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ce00 x1875449616512512/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575579.511136:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575579.511140:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575579.511141:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575579.511145:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7492 00000100:00080000:0.0:1788575579.511148:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575579.511152:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575579.511161:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575579.511173:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616512640/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575579.511184:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575579.511195:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575579.511197:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575579.511201:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7492 00000100:00080000:0.0:1788575579.511204:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575579.511208:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575579.511213:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575579.511222:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616512768/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575579.511365:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616512512:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575579.511449:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616512640:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575579.511475:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616512768:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575579.702923:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575580.023008:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575580.726957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575581.047079:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575581.750909:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575582.070972:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1788575582.583196:0:2372:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:0.0:1788575582.583248:0:2372:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1788575582.583265:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:0.0:1788575582.583310:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:0.0:1788575582.583319:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:3.0:1788575582.583356:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575579/real 1788575582] req@ffff9ada4557ce00 x1875449616512512/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575604 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575582.583388:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ce00 x1875449616512512/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575604 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575582.583400:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575582.583405:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575582.583409:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616512512:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575582.583417:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575579/real 1788575582] req@ffff9ada4557c700 x1875449616512640/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575604 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575582.583429:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616512640/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575604 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575582.583437:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575582.583441:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575582.583444:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616512640:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575582.583451:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575579/real 1788575582] req@ffff9ada4557ca80 x1875449616512768/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575604 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575582.583461:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616512768/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575604 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575582.583468:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575582.583471:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575582.583474:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616512768:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575582.774955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575583.095039:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575583.798953:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575584.118979:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575584.631254:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575584.631264:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575584.631268:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575584.631277:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7497 00000100:00080000:0.0:1788575584.631287:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575584.631294:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575584.631326:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616512896/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575584.631360:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575584.631364:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575584.631366:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575584.631369:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7497 00000100:00080000:0.0:1788575584.631373:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575584.631376:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575584.631384:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575584.631395:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616513024/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575584.631405:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575584.631408:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575584.631409:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575584.631413:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7497 00000100:00080000:0.0:1788575584.631416:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575584.631419:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575584.631423:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575584.631433:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616513152/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575584.631459:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616512896:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788575584.631519:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575584.631619:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575584/real 1788575584] req@ffff9ada4557d880 x1875449616512896/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575609 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575584.631639:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616512896/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575609 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575584.631649:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575584.631653:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575584.631657:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616512896:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575584.631664:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616513024:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575584.631686:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575584.631699:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575584/real 1788575584] req@ffff9ada4557dc00 x1875449616513024/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575609 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575584.631710:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616513024/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575609 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575584.631718:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575584.631721:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575584.631724:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616513024:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575584.631729:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616513152:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575584.631746:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575584.631757:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575584/real 1788575584] req@ffff9ada4557f800 x1875449616513152/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575609 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575584.631776:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616513152/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575609 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575584.631784:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575584.631787:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575584.631790:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616513152:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575584.822942:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575585.143008:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575585.846964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575586.166968:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575586.870931:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575587.190998:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575587.894953:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575588.215023:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575588.918937:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575589.239018:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575589.750984:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575589.750995:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575589.750999:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575589.751009:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7502 00000100:00080000:0.0:1788575589.751020:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575589.751027:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575589.751066:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616513280/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575589.751095:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575589.751097:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575589.751099:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575589.751101:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7502 00000100:00080000:0.0:1788575589.751104:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575589.751106:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575589.751116:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575589.751128:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616513408/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575589.751139:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575589.751143:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575589.751145:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575589.751149:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7502 00000100:00080000:0.0:1788575589.751154:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575589.751159:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575589.751163:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575589.751176:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616513536/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575589.751179:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616513280:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788575589.751261:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575589.751289:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575589/real 1788575589] req@ffff9ada4557e300 x1875449616513280/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575614 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575589.751303:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616513280/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575614 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575589.751311:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575589.751313:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575589.751316:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616513280:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575589.751321:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616513408:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575589.751334:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575589.751341:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575589/real 1788575589] req@ffff9ada4557fb80 x1875449616513408/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575614 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575589.751347:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616513408/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575614 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575589.751352:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575589.751354:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575589.751355:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616513408:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575589.751374:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616513536:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575589.751385:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575589.751390:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575589/real 1788575589] req@ffff9ada4557ed80 x1875449616513536/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575614 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575589.751396:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616513536/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575614 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575589.751401:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575589.751403:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575589.751405:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616513536:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575589.943069:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575590.263043:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575590.966998:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575591.287013:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575591.990969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575592.311054:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575593.014996:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575593.335085:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575594.038996:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575594.359089:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575594.870984:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575594.870997:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575594.871008:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575594.871018:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7507 00000100:00080000:1.0:1788575594.871029:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575594.871037:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575594.871076:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f800 x1875449616513664/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575594.871099:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575594.871102:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575594.871104:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575594.871107:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7507 00000100:00080000:1.0:1788575594.871109:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575594.871119:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575594.871132:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575594.871146:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566dc00 x1875449616513792/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575594.871152:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616513664:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788575594.871160:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575594.871167:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788575594.871169:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575594.871172:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7507 00000100:00080000:1.0:1788575594.871175:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575594.871178:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575594.871182:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575594.871192:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ed80 x1875449616513920/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575594.871243:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575594.871282:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575594/real 1788575594] req@ffff9ada4566f800 x1875449616513664/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575619 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575594.871297:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f800 x1875449616513664/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575619 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575594.871305:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575594.871308:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575594.871311:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616513664:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575594.871332:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616513792:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575594.871347:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575594.871354:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575594/real 1788575594] req@ffff9ada4566dc00 x1875449616513792/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575619 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575594.871361:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566dc00 x1875449616513792/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575619 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575594.871366:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575594.871368:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575594.871371:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616513792:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575594.871375:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616513920:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575594.871392:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575594.871403:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575594/real 1788575594] req@ffff9ada4566ed80 x1875449616513920/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575619 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575594.871414:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ed80 x1875449616513920/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575619 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575594.871422:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575594.871424:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575594.871426:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616513920:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575595.062998:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575595.383015:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575596.086980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575596.407085:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575597.110967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575597.431038:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575598.134953:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575598.454977:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575599.158929:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575599.479054:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575599.990991:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575599.990998:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575599.991001:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575599.991009:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7512 00000100:00080000:0.0:1788575599.991018:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575599.991024:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575599.991053:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616514048/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575599.991076:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575599.991080:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575599.991082:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575599.991086:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7512 00000100:00080000:0.0:1788575599.991089:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575599.991093:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575599.991102:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575599.991113:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d180 x1875449616514176/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575599.991123:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575599.991126:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575599.991127:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575599.991131:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7512 00000100:00080000:0.0:1788575599.991134:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575599.991138:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575599.991142:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575599.991151:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616514304/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575599.991182:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616514048:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788575599.991235:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575599.991261:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575599/real 1788575599] req@ffff9ada4557e680 x1875449616514048/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575624 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575599.991274:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616514048/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575624 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575599.991281:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575599.991283:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575599.991286:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616514048:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575599.991290:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616514176:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575599.991307:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575599.991318:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575599/real 1788575599] req@ffff9ada4557d180 x1875449616514176/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575624 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575599.991329:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d180 x1875449616514176/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575624 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575599.991338:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575599.991341:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575599.991344:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616514176:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575599.993408:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616514304:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575599.993445:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575599.993458:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575599/real 1788575599] req@ffff9ada4557ea00 x1875449616514304/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575624 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575599.993468:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616514304/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575624 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575599.993474:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575599.993476:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575599.993479:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616514304:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575600.183093:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575600.503000:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575601.206947:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575601.527032:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575602.230974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575602.551171:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575603.255007:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575603.575059:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575604.278969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575604.599030:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575605.110949:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575605.110958:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575605.110964:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575605.110974:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7517 00000100:00080000:2.0:1788575605.110983:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575605.110990:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575605.111024:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624700 x1875449616514432/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575605.111049:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575605.111053:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575605.111055:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575605.111059:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7517 00000100:00080000:2.0:1788575605.111064:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575605.111069:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575605.111079:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575605.111087:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616514432:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788575605.111096:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627100 x1875449616514560/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575605.111108:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575605.111112:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575605.111113:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575605.111117:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7517 00000100:00080000:2.0:1788575605.111121:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575605.111126:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575605.111131:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575605.111141:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616514688/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575605.111149:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575605.111183:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575605/real 1788575605] req@ffff9ada60624700 x1875449616514432/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575630 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575605.111196:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624700 x1875449616514432/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575630 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575605.111203:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575605.111206:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575605.111208:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616514432:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575605.111225:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616514560:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575605.111240:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575605.111248:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575605/real 1788575605] req@ffff9ada60627100 x1875449616514560/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575630 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575605.111255:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627100 x1875449616514560/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575630 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575605.111260:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575605.111262:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575605.111264:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616514560:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575605.111267:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616514688:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575605.111276:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575605.111282:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575605/real 1788575605] req@ffff9ada60626680 x1875449616514688/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575630 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575605.111288:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616514688/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575630 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575605.111292:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575605.111294:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575605.111295:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616514688:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575605.302914:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575605.623011:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575606.326960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575606.647020:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575607.352573:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575607.671009:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575608.375016:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575608.695077:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575609.399065:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575609.719191:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788575610.230977:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575610.230985:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788575610.230988:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575610.230996:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7522 00000100:00080000:3.0:1788575610.231004:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575610.231009:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575610.231038:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0380 x1875449616514816/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575610.231061:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575610.231065:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788575610.231066:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575610.231069:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7522 00000100:00080000:3.0:1788575610.231071:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575610.231073:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575610.231080:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575610.231087:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3480 x1875449616514944/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575610.231094:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575610.231096:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788575610.231097:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575610.231099:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7522 00000100:00080000:3.0:1788575610.231101:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575610.231103:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575610.231106:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575610.231115:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616514816:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575610.231115:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0700 x1875449616515072/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575610.231169:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575610.231195:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575610/real 1788575610] req@ffff9ada604e0380 x1875449616514816/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575635 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575610.231211:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0380 x1875449616514816/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575635 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575610.231221:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575610.231224:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575610.231228:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616514816:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575610.231235:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616514944:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575610.231255:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575610.231267:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575610/real 1788575610] req@ffff9ada604e3480 x1875449616514944/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575635 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575610.231277:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3480 x1875449616514944/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575635 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575610.231284:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575610.231286:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575610.231289:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616514944:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575610.231310:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616515072:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575610.231325:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575610.231333:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575610/real 1788575610] req@ffff9ada604e0700 x1875449616515072/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575635 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575610.231344:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0700 x1875449616515072/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575635 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575610.231351:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575610.231354:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575610.231357:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616515072:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575610.422974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575610.742962:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575611.446963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575611.767032:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575612.470957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575612.791043:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575613.495233:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575613.815078:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575614.518980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575614.839141:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575615.350996:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575615.351006:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575615.351010:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575615.351021:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7527 00000100:00080000:1.0:1788575615.351030:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575615.351037:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575615.351069:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c000 x1875449616515200/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575615.351096:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575615.351100:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575615.351101:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575615.351105:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7527 00000100:00080000:1.0:1788575615.351108:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575615.351111:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575615.351119:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575615.351130:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566df80 x1875449616515328/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575615.351140:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1788575615.351144:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616515200:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788575615.351144:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788575615.351147:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575615.351150:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7527 00000100:00080000:1.0:1788575615.351153:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575615.351158:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575615.351164:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575615.351174:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566fb80 x1875449616515456/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575615.351216:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575615.351245:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575615/real 1788575615] req@ffff9ada4566c000 x1875449616515200/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575640 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575615.351268:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c000 x1875449616515200/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575640 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575615.351275:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575615.351277:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575615.351280:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616515200:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575615.351296:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616515328:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575615.351319:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575615.351328:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575615/real 1788575615] req@ffff9ada4566df80 x1875449616515328/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575640 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575615.351340:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566df80 x1875449616515328/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575640 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575615.351354:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575615.351358:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575615.351361:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616515328:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575615.351367:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616515456:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575615.351387:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575615.351398:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575615/real 1788575615] req@ffff9ada4566fb80 x1875449616515456/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575640 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575615.351409:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566fb80 x1875449616515456/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575640 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575615.351431:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575615.351434:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575615.351437:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616515456:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575615.542961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575615.863019:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575616.567264:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575616.887197:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575617.591024:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575617.911093:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575618.614933:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575618.935078:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575619.639039:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575619.959049:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575620.470992:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575620.471001:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575620.471006:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575620.471016:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7533 00000100:00080000:1.0:1788575620.471027:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575620.471036:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575620.471086:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566fb80 x1875449616515584/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575620.471120:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575620.471123:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575620.471124:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575620.471127:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7533 00000100:00080000:1.0:1788575620.471130:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575620.471133:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575620.471143:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575620.471153:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566df80 x1875449616515712/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575620.471161:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575620.471164:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788575620.471166:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1788575620.471167:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616515584:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788575620.471171:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7533 00000100:00080000:1.0:1788575620.471175:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575620.471177:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575620.471181:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575620.471193:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c000 x1875449616515840/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575620.471237:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575620.471271:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575620/real 1788575620] req@ffff9ada4566fb80 x1875449616515584/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575645 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575620.471294:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566fb80 x1875449616515584/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575645 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575620.471302:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575620.471306:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575620.471310:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616515584:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575620.471337:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616515712:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575620.471358:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575620.471370:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575620/real 1788575620] req@ffff9ada4566df80 x1875449616515712/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575645 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575620.471382:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566df80 x1875449616515712/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575645 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575620.471390:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575620.471393:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575620.471397:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616515712:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575620.471404:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616515840:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575620.471423:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575620.471434:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575620/real 1788575620] req@ffff9ada4566c000 x1875449616515840/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575645 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575620.471445:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c000 x1875449616515840/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575645 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575620.471453:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575620.471456:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575620.471459:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616515840:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575620.662948:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575620.983078:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575621.686974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575622.007160:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575622.710969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575623.031019:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575623.734977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575624.055029:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575624.758970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575625.079025:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788575625.591027:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575625.591038:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788575625.591041:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575625.591049:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7538 00000100:00080000:3.0:1788575625.591058:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575625.591064:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575625.591109:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1180 x1875449616515968/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575625.591135:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575625.591141:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788575625.591142:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575625.591146:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7538 00000100:00080000:3.0:1788575625.591150:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575625.591153:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575625.591162:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575625.591175:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3800 x1875449616516096/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575625.591186:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1788575625.591190:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616515968:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788575625.591191:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788575625.591195:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575625.591203:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7538 00000100:00080000:3.0:1788575625.591207:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575625.591210:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575625.591215:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575625.591228:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1880 x1875449616516224/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575625.591268:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575625.591304:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575625/real 1788575625] req@ffff9ada604e1180 x1875449616515968/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575650 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575625.591322:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1180 x1875449616515968/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575650 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575625.591332:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575625.591336:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575625.591340:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616515968:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575625.591362:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616516096:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575625.591382:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575625.591393:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575625/real 1788575625] req@ffff9ada604e3800 x1875449616516096/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575650 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575625.591405:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3800 x1875449616516096/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575650 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575625.591413:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575625.591416:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575625.591419:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616516096:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575625.591424:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616516224:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575625.591584:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575625.591595:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575625/real 1788575625] req@ffff9ada604e1880 x1875449616516224/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575650 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575625.591606:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1880 x1875449616516224/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575650 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575625.591613:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575625.591615:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575625.591618:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616516224:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575625.782956:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575626.103036:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575626.806968:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575627.128097:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575627.830920:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575628.151037:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575628.854954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575629.174996:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575629.878976:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575630.199090:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575630.710991:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575630.711001:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575630.711004:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575630.711014:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7543 00000100:00080000:0.0:1788575630.711024:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575630.711031:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575630.711063:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d500 x1875449616516352/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575630.711089:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575630.711092:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575630.711094:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575630.711097:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7543 00000100:00080000:0.0:1788575630.711100:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575630.711104:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575630.711112:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575630.711125:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557df80 x1875449616516480/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575630.711125:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616516352:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788575630.711141:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575630.711144:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575630.711146:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575630.711150:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7543 00000100:00080000:0.0:1788575630.711153:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575630.711156:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575630.711161:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575630.711171:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616516608/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575630.711184:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575630.711213:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575630/real 1788575630] req@ffff9ada4557d500 x1875449616516352/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575655 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575630.711226:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d500 x1875449616516352/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575655 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575630.711233:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575630.711235:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575630.711238:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616516352:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575630.711257:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616516480:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575630.711278:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575630.711289:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575630/real 1788575630] req@ffff9ada4557df80 x1875449616516480/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575655 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575630.711301:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557df80 x1875449616516480/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575655 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575630.711309:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575630.711312:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575630.711314:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616516480:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575630.711318:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616516608:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575630.711331:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575630.711338:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575630/real 1788575630] req@ffff9ada4557ea00 x1875449616516608/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575655 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575630.711345:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616516608/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575655 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575630.711349:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575630.711351:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575630.711353:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616516608:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575630.902955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575631.223062:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575631.926980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575632.247048:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575632.951072:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575633.271080:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575633.974958:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575634.295049:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575634.998970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575635.318984:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575635.830969:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575635.830976:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575635.830979:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575635.830987:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7548 00000100:00080000:0.0:1788575635.830997:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575635.831004:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575635.831041:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d180 x1875449616516736/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575635.831065:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575635.831067:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575635.831068:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575635.831071:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7548 00000100:00080000:0.0:1788575635.831073:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575635.831076:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575635.831085:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575635.831098:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616516864/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575635.831110:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575635.831113:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575635.831117:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1788575635.831117:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616516736:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788575635.831125:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7548 00000100:00080000:0.0:1788575635.831131:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575635.831134:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575635.831139:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575635.831151:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616516992/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575635.831182:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575635.831215:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575635/real 1788575635] req@ffff9ada4557d180 x1875449616516736/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575660 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575635.831233:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d180 x1875449616516736/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575660 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575635.831243:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575635.831247:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575635.831251:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616516736:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575635.831275:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616516864:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575635.831295:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575635.831306:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575635/real 1788575635] req@ffff9ada4557e680 x1875449616516864/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575660 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575635.831317:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616516864/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575660 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575635.831326:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575635.831329:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575635.831332:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616516864:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575635.831338:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616516992:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575635.831354:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575635.831363:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575635/real 1788575635] req@ffff9ada4557ed80 x1875449616516992/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575660 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575635.831374:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616516992/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575660 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575635.831383:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575635.831386:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575635.831389:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616516992:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575636.022970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575636.343020:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575637.046971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575637.366985:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575638.070928:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575638.392345:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575639.094970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575639.415022:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575640.118962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575640.439194:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575640.951699:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575640.951708:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575640.951712:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575640.951723:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7553 00000100:00080000:0.0:1788575640.951734:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575640.951742:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575640.951789:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616517120/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575640.951906:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575640.951911:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575640.951913:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575640.951917:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7553 00000100:00080000:0.0:1788575640.951922:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575640.951927:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575640.951937:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575640.951953:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616517248/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575640.951966:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1788575640.951967:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616517120:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788575640.951976:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575640.951980:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575640.951985:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7553 00000100:00080000:0.0:1788575640.951989:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575640.951995:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575640.952000:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575640.952012:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616517376/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575640.952033:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575640.952068:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575640/real 1788575640] req@ffff9ada4557fb80 x1875449616517120/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575665 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575640.952086:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616517120/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575665 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575640.952095:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575640.952099:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575640.952102:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616517120:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575640.952129:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616517248:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575640.952151:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575640.952163:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575640/real 1788575640] req@ffff9ada4557e300 x1875449616517248/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575665 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575640.952174:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616517248/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575665 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575640.952182:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575640.952185:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575640.952188:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616517248:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575640.952196:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616517376:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575640.952214:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575640.952225:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575640/real 1788575640] req@ffff9ada4557f800 x1875449616517376/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575665 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575640.952236:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616517376/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575665 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575640.952249:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575640.952252:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575640.952255:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616517376:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575641.142992:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575641.463013:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575642.166970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575642.486967:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575643.190976:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575643.511036:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575644.215012:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575644.535049:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575645.238940:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575645.559007:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575646.070995:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575646.071005:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575646.071009:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575646.071017:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7558 00000100:00080000:2.0:1788575646.071025:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575646.071031:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575646.071061:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616517504/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575646.071084:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575646.071087:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575646.071088:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575646.071092:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7558 00000100:00080000:2.0:1788575646.071095:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575646.071099:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575646.071108:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575646.071121:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616517632/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575646.071124:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616517504:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788575646.071142:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575646.071146:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575646.071148:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575646.071151:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7558 00000100:00080000:2.0:1788575646.071154:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575646.071157:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575646.071162:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575646.071170:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616517760/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575646.071230:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616517632:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575646.071265:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616517760:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575646.262970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575646.583032:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575647.286931:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575647.607039:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575648.310992:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575648.631033:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1788575649.143133:0:2373:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:1.0:1788575649.143181:0:2373:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1788575649.143197:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:1.0:1788575649.143238:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:1.0:1788575649.143248:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:3.0:1788575649.143288:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575646/real 1788575649] req@ffff9ada60626a00 x1875449616517632/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575671 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575649.143328:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616517632/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575671 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575649.143343:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575649.143348:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575649.143352:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616517632:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575649.143360:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575646/real 1788575649] req@ffff9ada60625500 x1875449616517760/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575671 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575649.143373:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616517760/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575671 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575649.143380:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575649.143384:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575649.143387:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616517760:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575649.143394:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575646/real 1788575649] req@ffff9ada60624380 x1875449616517504/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575671 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575649.143407:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616517504/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575671 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575649.143432:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575649.143436:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575649.143439:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616517504:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575649.335038:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575649.655053:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575650.358974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575650.679010:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575651.190973:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575651.190983:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575651.190987:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575651.190996:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7563 00000100:00080000:0.0:1788575651.191006:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575651.191013:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575651.191043:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616517888/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575651.191070:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575651.191074:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575651.191076:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575651.191080:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7563 00000100:00080000:0.0:1788575651.191083:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575651.191087:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575651.191096:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575651.191109:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616518016/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575651.191120:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575651.191125:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1788575651.191126:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616517888:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788575651.191128:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575651.191137:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7563 00000100:00080000:0.0:1788575651.191141:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575651.191145:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575651.191150:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575651.191163:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616518144/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575651.191200:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575651.191231:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575651/real 1788575651] req@ffff9ada4557dc00 x1875449616517888/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575676 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575651.191249:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616517888/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575676 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575651.191260:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575651.191264:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575651.191268:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616517888:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575651.191290:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616518016:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575651.191310:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575651.191321:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575651/real 1788575651] req@ffff9ada4557d880 x1875449616518016/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575676 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575651.191332:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616518016/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575676 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575651.191346:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575651.191349:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575651.191352:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616518016:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575651.191357:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616518144:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575651.191373:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575651.191381:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575651/real 1788575651] req@ffff9ada4557ca80 x1875449616518144/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575676 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575651.191391:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616518144/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575676 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575651.191398:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575651.191401:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575651.191404:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616518144:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575651.382964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575651.703019:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575652.406958:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575652.727101:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575653.430967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575653.751087:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575654.454996:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575654.774995:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575655.478971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575655.799082:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575656.310964:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575656.310976:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575656.310980:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575656.310990:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7568 00000100:00080000:0.0:1788575656.311001:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575656.311008:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575656.311050:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616518272/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575656.311081:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575656.311085:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575656.311087:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575656.311090:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7568 00000100:00080000:0.0:1788575656.311094:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575656.311097:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575656.311106:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575656.311119:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ce00 x1875449616518400/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575656.311134:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575656.311137:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575656.311139:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575656.311143:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7568 00000100:00080000:0.0:1788575656.311146:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575656.311152:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575656.311154:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616518272:192.168.202.113@tcp:250:kworker.0 00000008:00100000:0.0:1788575656.311158:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575656.311172:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616518528/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575656.311225:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575656.311261:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575656/real 1788575656] req@ffff9ada4557c700 x1875449616518272/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575681 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575656.311282:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616518272/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575681 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575656.311292:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575656.311296:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575656.311300:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616518272:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575656.311307:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616518400:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575656.311330:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575656.311343:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575656/real 1788575656] req@ffff9ada4557ce00 x1875449616518400/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575681 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575656.311355:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ce00 x1875449616518400/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575681 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575656.311363:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575656.311366:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575656.311369:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616518400:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575656.311393:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616518528:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575656.311413:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575656.311425:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575656/real 1788575656] req@ffff9ada4557f480 x1875449616518528/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575681 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575656.311442:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616518528/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575681 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575656.311450:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575656.311453:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575656.311456:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616518528:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575656.502969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575656.823026:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575657.526971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575657.847100:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575658.551022:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575658.871051:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575659.574978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575659.895399:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575660.599348:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575660.919085:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575661.431019:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575661.431034:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575661.431039:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575661.431052:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7573 00000100:00080000:0.0:1788575661.431065:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575661.431074:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575661.431123:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c380 x1875449616518656/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575661.431159:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575661.431164:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575661.431166:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575661.431170:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7573 00000100:00080000:0.0:1788575661.431173:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575661.431178:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575661.431206:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616518656:192.168.202.113@tcp:250:kworker.0 00000008:00100000:0.0:1788575661.431279:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1788575661.431287:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575661.431322:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c000 x1875449616518784/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575661.431338:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575661/real 1788575661] req@ffff9ada4557c380 x1875449616518656/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575686 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575661.431343:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575661.431354:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575661.431357:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575661.431368:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7573 00000100:00100000:3.0:1788575661.431368:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c380 x1875449616518656/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575686 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575661.431380:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575661.431387:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575661.431399:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00080000:3.0:1788575661.431406:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575661.431415:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575661.431419:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616518912/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575661.431424:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616518656:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575661.431461:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616518784:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575661.431486:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575661.431502:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575661/real 1788575661] req@ffff9ada4557c000 x1875449616518784/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575686 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575661.431516:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c000 x1875449616518784/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575686 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575661.431525:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575661.431528:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575661.431531:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616518784:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575661.431539:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616518912:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575661.431572:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575661.431584:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575661/real 1788575661] req@ffff9ada4557f480 x1875449616518912/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575686 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575661.431593:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616518912/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575686 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575661.431610:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575661.431614:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575661.431617:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616518912:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575661.623014:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575661.943070:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575662.647002:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575662.967096:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575663.670991:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575663.991089:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575664.695015:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575665.015236:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575665.718970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575666.039088:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575666.551030:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575666.551040:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575666.551044:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575666.551055:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7579 00000100:00080000:1.0:1788575666.551065:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575666.551071:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575666.551111:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ed80 x1875449616519040/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575666.551142:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575666.551146:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575666.551148:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575666.551151:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7579 00000100:00080000:1.0:1788575666.551155:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575666.551158:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575666.551168:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575666.551180:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566dc00 x1875449616519168/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575666.551192:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575666.551196:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788575666.551197:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575666.551206:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7579 00000100:00100000:3.0:1788575666.551210:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616519040:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788575666.551211:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575666.551224:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575666.551230:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575666.551247:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f800 x1875449616519296/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575666.551305:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575666.551363:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575666/real 1788575666] req@ffff9ada4566ed80 x1875449616519040/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575691 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575666.551385:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ed80 x1875449616519040/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575691 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575666.551395:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575666.551399:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575666.551407:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616519040:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575666.552682:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616519168:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575666.552734:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575666.552755:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575666/real 1788575666] req@ffff9ada4566dc00 x1875449616519168/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575691 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575666.552768:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566dc00 x1875449616519168/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575691 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575666.552775:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575666.552778:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575666.552781:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616519168:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575666.552789:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616519296:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575666.552822:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575666.552834:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575666/real 1788575666] req@ffff9ada4566f800 x1875449616519296/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575691 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575666.552846:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f800 x1875449616519296/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575691 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575666.552869:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575666.552873:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575666.552876:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616519296:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575666.743010:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575667.063192:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575667.766993:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575668.087027:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575668.790993:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575669.111122:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575669.815003:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575670.135026:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575670.838975:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575671.159066:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575671.670978:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575671.670992:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575671.670997:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575671.671008:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7584 00000100:00080000:1.0:1788575671.671021:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575671.671030:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575671.671076:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e300 x1875449616519424/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575671.671110:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575671.671113:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575671.671115:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575671.671119:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7584 00000100:00080000:1.0:1788575671.671123:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575671.671127:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575671.671140:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575671.671155:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f480 x1875449616519552/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575671.671162:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616519424:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788575671.671168:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575671.671173:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788575671.671174:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575671.671179:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7584 00000100:00080000:1.0:1788575671.671182:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575671.671186:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575671.671191:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575671.671202:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ea00 x1875449616519680/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575671.671250:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575671.671288:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575671/real 1788575671] req@ffff9ada4566e300 x1875449616519424/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575696 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575671.671307:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e300 x1875449616519424/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575696 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575671.671423:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575671.671430:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575671.671435:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616519424:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575671.671464:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616519552:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575671.671521:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575671.671544:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575671/real 1788575671] req@ffff9ada4566f480 x1875449616519552/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575696 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575671.671560:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f480 x1875449616519552/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575696 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575671.671582:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575671.671586:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575671.671589:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616519552:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575671.671596:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616519680:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575671.671615:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575671.671623:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575671/real 1788575671] req@ffff9ada4566ea00 x1875449616519680/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575696 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575671.671631:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ea00 x1875449616519680/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575696 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575671.671635:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575671.671638:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575671.671640:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616519680:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575671.862974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575672.183034:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575672.886942:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575673.207072:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575673.910997:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575674.231084:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575674.934961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575675.255067:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575675.958995:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575676.279848:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575676.791006:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575676.791018:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575676.791022:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575676.791031:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7589 00000100:00080000:2.0:1788575676.791043:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575676.791050:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575676.791096:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616519808/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575676.791127:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575676.791131:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575676.791133:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575676.791137:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7589 00000100:00080000:2.0:1788575676.791141:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575676.791146:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575676.791157:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575676.791174:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616519936/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575676.791174:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616519808:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788575676.791190:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575676.791193:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575676.791195:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575676.791199:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7589 00000100:00080000:2.0:1788575676.791203:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575676.791207:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575676.791213:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575676.791224:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616520064/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575676.791264:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575676.791304:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575676/real 1788575676] req@ffff9ada60624a80 x1875449616519808/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575701 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575676.791323:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616519808/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575701 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575676.791344:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575676.791349:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575676.791352:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616519808:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575676.791383:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616519936:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575676.791407:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575676.791420:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575676/real 1788575676] req@ffff9ada60625880 x1875449616519936/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575701 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575676.791432:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616519936/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575701 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575676.791446:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575676.791449:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575676.791452:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616519936:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575676.791457:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616520064:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575676.791488:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575676.791499:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575676/real 1788575676] req@ffff9ada60625c00 x1875449616520064/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575701 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575676.791510:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616520064/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575701 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575676.791518:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575676.791521:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575676.791525:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616520064:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575676.983003:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575677.303082:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575678.007108:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575678.327016:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575679.030977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575679.351002:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575680.054987:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575680.375116:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575681.079023:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575681.399104:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575681.911055:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575681.911071:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575681.911078:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575681.911092:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7594 00000100:00080000:0.0:1788575681.911111:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575681.911121:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575681.911177:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ce00 x1875449616520192/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575681.911221:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575681.911226:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575681.911228:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575681.911232:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7594 00000100:00080000:0.0:1788575681.911236:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575681.911240:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575681.911253:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575681.911268:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616520320/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575681.911280:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1788575681.911282:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616520192:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788575681.911285:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575681.911289:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575681.911298:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7594 00000100:00080000:0.0:1788575681.911302:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575681.911306:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575681.911311:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575681.911323:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616520448/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575681.911387:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575681.911450:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575681/real 1788575681] req@ffff9ada4557ce00 x1875449616520192/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575706 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575681.911483:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ce00 x1875449616520192/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575706 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575681.911496:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575681.911501:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575681.911505:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616520192:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575681.911552:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616520320:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575681.911576:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575681.911590:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575681/real 1788575681] req@ffff9ada4557c700 x1875449616520320/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575706 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575681.911611:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616520320/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575706 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575681.911620:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575681.911624:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575681.911627:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616520320:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575681.911634:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616520448:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575681.911655:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575681.911666:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575681/real 1788575681] req@ffff9ada4557ca80 x1875449616520448/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575706 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575681.911684:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616520448/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575706 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575681.911692:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575681.911695:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575681.911698:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616520448:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575682.102977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575682.423107:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575683.127033:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575683.447093:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575684.150999:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575684.471091:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575685.175001:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575685.495109:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575686.199003:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575686.519290:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575687.030963:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575687.030973:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575687.030977:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575687.030987:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7599 00000100:00080000:0.0:1788575687.030997:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575687.031003:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575687.031035:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616520576/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575687.031061:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575687.031064:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575687.031065:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575687.031068:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7599 00000100:00080000:0.0:1788575687.031070:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575687.031072:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575687.031081:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575687.031092:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616520704/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575687.031101:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575687.031104:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575687.031107:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575687.031111:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7599 00000100:00080000:0.0:1788575687.031114:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575687.031119:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575687.031122:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616520576:192.168.202.113@tcp:250:kworker.0 00000008:00100000:0.0:1788575687.031125:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575687.031138:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616520832/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575687.031183:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575687.031216:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575687/real 1788575687] req@ffff9ada4557d880 x1875449616520576/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575712 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575687.031229:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616520576/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575712 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575687.031236:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575687.031239:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575687.031242:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616520576:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575687.031247:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616520704:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575687.031263:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575687.031270:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575687/real 1788575687] req@ffff9ada4557dc00 x1875449616520704/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575712 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575687.031277:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616520704/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575712 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575687.031281:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575687.031283:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575687.031285:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616520704:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575687.031301:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616520832:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575687.031310:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575687.031315:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575687/real 1788575687] req@ffff9ada4557f800 x1875449616520832/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575712 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575687.031322:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616520832/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575712 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575687.031326:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575687.031329:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575687.031331:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616520832:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575687.222958:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575687.543091:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575688.246948:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575688.567064:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575689.270954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575689.590997:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575690.294952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575690.615032:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575691.318971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575691.641080:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575692.151004:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575692.151015:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575692.151019:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575692.151028:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7604 00000100:00080000:2.0:1788575692.151039:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575692.151047:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575692.151093:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616520960/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575692.151121:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575692.151125:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575692.151127:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575692.151131:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7604 00000100:00080000:2.0:1788575692.151136:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575692.151140:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575692.151150:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575692.151150:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616520960:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788575692.151180:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616521088/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575692.151195:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575692.151200:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575692.151202:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575692.151207:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7604 00000100:00080000:2.0:1788575692.151223:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575692.151229:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000800:00000100:3.0:1788575692.151235:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000008:00100000:2.0:1788575692.151240:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575692.151258:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616521216/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575692.151290:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575692/real 1788575692] req@ffff9ada60624e00 x1875449616520960/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575717 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575692.151315:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616520960/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575717 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575692.151327:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575692.151332:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575692.151336:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616520960:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575692.151365:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616521088:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575692.151391:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575692.151403:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575692/real 1788575692] req@ffff9ada60627480 x1875449616521088/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575717 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575692.151416:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616521088/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575717 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575692.151424:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575692.151427:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575692.151431:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616521088:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575692.151437:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616521216:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575692.151457:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575692.151468:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575692/real 1788575692] req@ffff9ada60625f80 x1875449616521216/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575717 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575692.151479:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616521216/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575717 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575692.151487:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575692.151490:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575692.151493:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616521216:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575692.342966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575692.663016:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575693.367013:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575693.687063:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575694.390952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575694.711049:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575695.414991:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575695.735099:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575696.438955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575696.759115:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575697.270997:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575697.271007:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575697.271011:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575697.271020:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7609 00000100:00080000:2.0:1788575697.271032:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575697.271039:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575697.271084:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616521344/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575697.271114:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575697.271118:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575697.271120:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575697.271123:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7609 00000100:00080000:2.0:1788575697.271127:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575697.271131:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575697.271153:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575697.271166:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616521472/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575697.271176:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575697.271179:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575697.271181:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575697.271184:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7609 00000100:00080000:2.0:1788575697.271188:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575697.271193:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575697.271198:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575697.271209:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616521600/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575697.271219:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616521344:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788575697.271295:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575697.271329:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575697/real 1788575697] req@ffff9ada60625180 x1875449616521344/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575722 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575697.271361:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616521344/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575722 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575697.271379:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575697.271383:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575697.271387:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616521344:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575697.271394:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616521472:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575697.271418:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575697.271431:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575697/real 1788575697] req@ffff9ada60626d80 x1875449616521472/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575722 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575697.271442:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616521472/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575722 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575697.271450:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575697.271454:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575697.271457:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616521472:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575697.271493:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616521600:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575697.271512:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575697.271522:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575697/real 1788575697] req@ffff9ada60627b80 x1875449616521600/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575722 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575697.271533:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616521600/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575722 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575697.271540:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575697.271543:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575697.271546:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616521600:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575697.462979:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575697.783037:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575698.489929:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575698.806991:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575699.510927:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575699.831066:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575700.534966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575700.855106:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575701.558985:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575701.879095:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575702.390970:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575702.390980:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575702.390985:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575702.390995:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7614 00000100:00080000:1.0:1788575702.391009:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575702.391015:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575702.391047:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d500 x1875449616521728/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575702.391072:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575702.391076:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575702.391077:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575702.391081:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7614 00000100:00080000:1.0:1788575702.391085:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575702.391090:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575702.391099:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575702.391112:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ca80 x1875449616521856/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575702.391118:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616521728:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788575702.391129:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575702.391133:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788575702.391134:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575702.391139:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7614 00000100:00080000:1.0:1788575702.391143:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575702.391146:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575702.391151:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575702.391160:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d180 x1875449616521984/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575702.391179:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575702.391212:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575702/real 1788575702] req@ffff9ada4566d500 x1875449616521728/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575727 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575702.391230:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d500 x1875449616521728/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575727 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575702.391240:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575702.391243:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575702.391247:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616521728:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575702.391269:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616521856:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575702.391291:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575702.391304:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575702/real 1788575702] req@ffff9ada4566ca80 x1875449616521856/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575727 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575702.391316:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ca80 x1875449616521856/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575727 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575702.391324:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575702.391328:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575702.391331:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616521856:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575702.391337:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616521984:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575702.391355:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575702.391366:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575702/real 1788575702] req@ffff9ada4566d180 x1875449616521984/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575727 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575702.391377:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d180 x1875449616521984/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575727 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575702.391385:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575702.391388:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575702.391391:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616521984:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575702.582968:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575702.903028:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575703.606972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575703.927015:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575704.630974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575704.951052:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575705.654977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575705.975049:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575706.679008:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575706.999084:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788575707.511999:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575707.512012:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788575707.512017:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575707.512030:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7620 00000100:00080000:3.0:1788575707.512044:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575707.512054:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575707.512106:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2d80 x1875449616522112/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788575707.512194:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616522112:192.168.202.113@tcp:250:kworker.0 00000800:00000100:0.0:1788575707.512270:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00080000:3.0:1788575707.512307:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575707.512311:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788575707.512313:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575707.512318:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7620 00000100:00080000:3.0:1788575707.512322:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575707.512326:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575707.512337:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575707.512350:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0a80 x1875449616522240/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788575707.513665:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575707/real 1788575707] req@ffff9ada604e2d80 x1875449616522112/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575732 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575707.513708:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2d80 x1875449616522112/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575732 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575707.513720:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575707.513725:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575707.513729:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616522112:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575707.513761:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616522240:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575707.513811:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575707.513826:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575707/real 1788575707] req@ffff9ada604e0a80 x1875449616522240/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575732 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575707.513838:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0a80 x1875449616522240/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575732 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575707.513847:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575707.513850:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575707.513870:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616522240:192.168.202.113@tcp:38:kworker.0 00000100:00080000:3.0:1788575707.514529:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575707.514535:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788575707.514538:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575707.514544:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7620 00000100:00080000:3.0:1788575707.514549:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575707.514555:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575707.514566:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575707.514587:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2300 x1875449616522368/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788575707.514636:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616522368:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575707.514667:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575707.515611:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575707/real 1788575707] req@ffff9ada604e2300 x1875449616522368/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575732 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575707.515631:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2300 x1875449616522368/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575732 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575707.515647:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575707.515652:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575707.515655:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616522368:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575707.703066:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575708.023149:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575708.726956:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575709.048400:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575709.751043:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575710.071084:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575710.774978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575711.095050:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575711.798920:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575712.118999:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575712.631112:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575712.631123:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575712.631136:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575712.631146:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7625 00000100:00080000:1.0:1788575712.631157:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575712.631166:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575712.631202:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f100 x1875449616522496/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788575712.631288:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616522496:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788575712.631511:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575712.631516:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575712.631518:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575712.631523:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7625 00000100:00080000:1.0:1788575712.631528:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575712.631533:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575712.631542:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575712.631555:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c380 x1875449616522624/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788575712.631695:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616522624:192.168.202.113@tcp:38:kworker.0 00000100:00080000:1.0:1788575712.632202:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575712.632207:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788575712.632209:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575712.632213:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7625 00000100:00080000:1.0:1788575712.632218:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575712.632222:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575712.632228:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575712.632242:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c700 x1875449616522752/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788575712.632510:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616522752:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575712.822987:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575713.144888:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575713.847017:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575714.167078:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575714.871066:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575715.191060:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1788575715.703145:0:2371:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:2.0:1788575715.703186:0:2371:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1788575715.703616:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:0.0:1788575715.703746:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575712/real 1788575715] req@ffff9ada4566f100 x1875449616522496/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575737 ref 2 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575715.703770:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f100 x1875449616522496/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575737 ref 2 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575715.703792:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575715.703797:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575715.703802:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616522496:192.168.202.113@tcp:250:kworker.0 00000800:00000100:2.0:1788575715.703918:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788575715.703938:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:0.0:1788575715.704027:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575712/real 1788575715] req@ffff9ada4566c700 x1875449616522752/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575737 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575715.704045:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c700 x1875449616522752/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575737 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575715.704054:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575715.704059:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575715.704062:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616522752:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575715.704069:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575712/real 1788575715] req@ffff9ada4566c380 x1875449616522624/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575737 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575715.704189:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c380 x1875449616522624/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575737 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575715.704198:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575715.704202:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575715.704205:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616522624:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575715.894966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575716.215047:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575716.919222:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575717.240532:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575717.751022:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575717.751032:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575717.751036:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575717.751045:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7630 00000100:00080000:2.0:1788575717.751055:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575717.751061:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575717.751090:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626300 x1875449616522880/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575717.751124:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575717.751128:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575717.751130:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575717.751133:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7630 00000100:00080000:2.0:1788575717.751137:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575717.751140:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575717.751149:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575717.751165:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627800 x1875449616523008/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575717.751175:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575717.751178:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575717.751180:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575717.751183:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7630 00000100:00080000:2.0:1788575717.751186:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575717.751189:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575717.751194:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575717.751203:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616523136/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788575717.751333:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616522880:192.168.202.113@tcp:250:kworker.0 00000800:00000100:0.0:1788575717.751386:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575717.751409:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575717/real 1788575717] req@ffff9ada60626300 x1875449616522880/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575742 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575717.751420:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626300 x1875449616522880/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575742 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575717.751426:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575717.751429:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575717.751431:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616522880:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575717.751437:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616523008:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575717.751449:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575717.751455:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575717/real 1788575717] req@ffff9ada60627800 x1875449616523008/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575742 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575717.751462:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627800 x1875449616523008/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575742 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575717.751466:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575717.751468:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575717.751469:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616523008:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575717.751472:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616523136:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575717.751481:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575717.751485:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575717/real 1788575717] req@ffff9ada60627b80 x1875449616523136/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575742 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575717.751492:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616523136/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575742 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575717.751498:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575717.751500:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575717.751503:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616523136:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575717.942971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575718.263056:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575718.966968:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575719.287025:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575719.990954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575720.311039:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575721.014941:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575721.335038:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575722.038934:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575722.359008:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575722.870945:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575722.870954:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575722.870959:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575722.870968:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7635 00000100:00080000:2.0:1788575722.870978:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575722.870985:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575722.871026:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616523264/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575722.871064:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575722.871067:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575722.871069:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575722.871073:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7635 00000100:00080000:2.0:1788575722.871076:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575722.871080:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575722.871090:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575722.871103:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616523392/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788575722.871112:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616523264:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788575722.871116:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575722.871121:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575722.871123:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575722.871127:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7635 00000100:00080000:2.0:1788575722.871131:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575722.871135:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575722.871140:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575722.871152:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616523520/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575722.871174:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575722.871203:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575722/real 1788575722] req@ffff9ada60626d80 x1875449616523264/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575747 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575722.871230:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616523264/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575747 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575722.871239:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575722.871242:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575722.871245:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616523264:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575722.871262:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616523392:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575722.871284:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575722.871295:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575722/real 1788575722] req@ffff9ada60625180 x1875449616523392/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575747 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575722.871307:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616523392/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575747 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575722.871315:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575722.871318:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575722.871321:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616523392:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575722.871331:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616523520:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575722.871345:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575722.871352:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575722/real 1788575722] req@ffff9ada60625f80 x1875449616523520/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575747 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575722.871362:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616523520/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575747 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575722.871367:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575722.871369:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575722.871371:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616523520:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575723.062962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575723.382971:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575724.086940:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575724.407010:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575725.110969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575725.431082:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575726.134962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575726.455068:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575727.158930:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575727.479018:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575727.990974:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575727.990984:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575727.990987:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575727.990997:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7640 00000100:00080000:2.0:1788575727.991008:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575727.991015:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575727.991049:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616523648/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575727.991083:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575727.991092:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575727.991094:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575727.991098:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7640 00000100:00080000:2.0:1788575727.991101:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575727.991105:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575727.991114:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575727.991126:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616523776/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575727.991137:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1788575727.991141:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616523648:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788575727.991142:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575727.991146:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575727.991152:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7640 00000100:00080000:2.0:1788575727.991155:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575727.991159:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575727.991164:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575727.991177:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616523904/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575727.991210:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575727.991245:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575727/real 1788575727] req@ffff9ada60627480 x1875449616523648/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575752 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575727.991259:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616523648/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575752 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575727.991266:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575727.991268:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575727.991271:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616523648:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575727.991288:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616523776:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575727.991302:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575727.991309:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575727/real 1788575727] req@ffff9ada60624e00 x1875449616523776/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575752 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575727.991321:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616523776/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575752 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575727.991325:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575727.991327:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575727.991329:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616523776:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575727.991332:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616523904:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575727.991340:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575727.991345:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575727/real 1788575727] req@ffff9ada60625c00 x1875449616523904/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575752 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575727.991358:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616523904/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575752 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575727.991362:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575727.991364:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575727.991368:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616523904:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575728.182980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575728.502994:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575729.206948:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575729.527018:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575730.230996:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575730.551217:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575731.254994:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575731.575049:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575732.278966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575732.599033:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788575733.112718:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575733.112728:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788575733.112731:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575733.112741:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7645 00000100:00080000:3.0:1788575733.112750:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575733.112757:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575733.112792:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0000 x1875449616524032/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575733.112827:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575733.112830:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788575733.112832:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575733.112836:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7645 00000100:00080000:3.0:1788575733.112839:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575733.112843:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575733.112878:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616524032:192.168.202.113@tcp:250:kworker.0 00000008:00100000:3.0:1788575733.112883:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575733.112895:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1c00 x1875449616524160/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575733.112903:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575733.112906:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788575733.112908:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575733.112911:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7645 00000100:00080000:3.0:1788575733.112913:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575733.112916:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575733.112920:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575733.112927:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3b80 x1875449616524288/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575733.112932:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575733.114090:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575733/real 1788575733] req@ffff9ada604e0000 x1875449616524032/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575758 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575733.114274:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0000 x1875449616524032/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575758 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575733.114284:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575733.114289:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575733.114292:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616524032:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575733.114317:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616524160:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575733.114345:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575733.114360:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575733/real 1788575733] req@ffff9ada604e1c00 x1875449616524160/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575758 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575733.114373:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1c00 x1875449616524160/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575758 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575733.114382:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575733.114386:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575733.114389:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616524160:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575733.114395:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616524288:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575733.114415:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575733.114426:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575733/real 1788575733] req@ffff9ada604e3b80 x1875449616524288/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575758 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575733.114437:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3b80 x1875449616524288/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575758 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575733.114451:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575733.114454:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575733.114457:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616524288:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575733.303713:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575733.623122:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575734.326963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575734.647168:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575735.350941:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575735.671059:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575736.374947:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575736.695367:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575737.398965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575737.719038:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788575738.230942:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575738.230952:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788575738.230956:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575738.230966:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7650 00000100:00080000:3.0:1788575738.230976:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575738.230982:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575738.231014:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1f80 x1875449616524416/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575738.231042:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575738.231046:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788575738.231048:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575738.231055:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7650 00000100:00080000:3.0:1788575738.231059:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575738.231063:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575738.231072:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575738.231087:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2680 x1875449616524544/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575738.231097:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1788575738.231099:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616524416:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788575738.231102:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788575738.231106:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575738.231111:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7650 00000100:00080000:3.0:1788575738.231114:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575738.231118:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575738.231123:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575738.231135:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3100 x1875449616524672/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575738.231165:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575738.231198:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575738/real 1788575738] req@ffff9ada604e1f80 x1875449616524416/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575763 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575738.231217:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1f80 x1875449616524416/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575763 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575738.231227:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575738.231231:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575738.231235:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616524416:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575738.231256:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616524544:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575738.231277:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575738.231288:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575738/real 1788575738] req@ffff9ada604e2680 x1875449616524544/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575763 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575738.231299:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2680 x1875449616524544/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575763 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575738.231313:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575738.231316:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575738.231319:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616524544:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575738.231324:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616524672:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575738.231340:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575738.231349:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575738/real 1788575738] req@ffff9ada604e3100 x1875449616524672/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575763 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575738.231359:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3100 x1875449616524672/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575763 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575738.231366:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575738.231369:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575738.231377:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616524672:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575738.422970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575738.743010:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575739.446954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575739.767108:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575740.470964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575740.791001:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575741.494977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575741.814998:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575742.518965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575742.838994:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788575743.350938:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575743.350947:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788575743.350952:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575743.350961:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7655 00000100:00080000:3.0:1788575743.350970:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575743.350976:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575743.351016:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2a00 x1875449616524800/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575743.351044:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575743.351048:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788575743.351049:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575743.351053:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7655 00000100:00080000:3.0:1788575743.351057:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575743.351060:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575743.351068:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575743.351081:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1500 x1875449616524928/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575743.351091:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1788575743.351096:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616524800:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788575743.351097:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788575743.351102:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575743.351108:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7655 00000100:00080000:3.0:1788575743.351112:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575743.351115:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575743.351121:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575743.351134:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0e00 x1875449616525056/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575743.351147:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575743.351173:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575743/real 1788575743] req@ffff9ada604e2a00 x1875449616524800/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575768 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575743.351189:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2a00 x1875449616524800/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575768 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575743.351196:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575743.351198:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575743.351201:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616524800:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575743.351217:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616524928:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575743.351229:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575743.351236:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575743/real 1788575743] req@ffff9ada604e1500 x1875449616524928/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575768 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575743.351242:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1500 x1875449616524928/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575768 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575743.351247:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575743.351248:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575743.351250:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616524928:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575743.351257:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616525056:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575743.351267:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575743.351272:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575743/real 1788575743] req@ffff9ada604e0e00 x1875449616525056/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575768 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575743.351279:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0e00 x1875449616525056/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575768 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575743.351287:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575743.351289:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575743.351292:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616525056:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575743.542945:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575743.862959:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575744.566949:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575744.887040:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575745.590963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575745.911134:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575746.614977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575746.935061:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788575747.638987:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575747.959046:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575748.470988:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575748.470996:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575748.471000:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575748.471009:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7661 00000100:00080000:1.0:1788575748.471018:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575748.471024:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575748.471060:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ce00 x1875449616525184/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575748.471086:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575748.471090:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575748.471091:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575748.471095:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7661 00000100:00080000:1.0:1788575748.471099:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575748.471103:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575748.471111:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575748.471126:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d880 x1875449616525312/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575748.471136:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575748.471140:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:0.0:1788575748.471144:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616525184:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788575748.471144:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575748.471152:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7661 00000100:00080000:1.0:1788575748.471156:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575748.471161:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575748.471165:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575748.471177:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e680 x1875449616525440/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575748.471237:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575748.471272:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575748/real 1788575748] req@ffff9ada4566ce00 x1875449616525184/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575773 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575748.471288:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ce00 x1875449616525184/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575773 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575748.471305:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575748.471312:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575748.471315:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616525184:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575748.471336:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616525312:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575748.471357:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575748.471368:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575748/real 1788575748] req@ffff9ada4566d880 x1875449616525312/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575773 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575748.471384:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d880 x1875449616525312/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575773 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575748.471391:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575748.471394:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575748.471398:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616525312:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575748.471403:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616525440:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575748.471418:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575748.471427:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575748/real 1788575748] req@ffff9ada4566e680 x1875449616525440/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575773 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575748.471437:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e680 x1875449616525440/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575773 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575748.471444:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575748.471447:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575748.471450:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616525440:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575748.662954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575748.983108:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575749.686978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575750.007047:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575750.710970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575751.031043:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575751.734968:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575752.055040:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575752.758977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575753.079036:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788575753.591012:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575753.591022:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788575753.591026:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575753.591036:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7666 00000100:00080000:3.0:1788575753.591045:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575753.591053:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575753.591096:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0e00 x1875449616525568/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575753.591117:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575753.591120:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788575753.591121:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575753.591125:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7666 00000100:00080000:3.0:1788575753.591139:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575753.591144:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575753.591153:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575753.591172:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1500 x1875449616525696/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788575753.591173:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616525568:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788575753.591190:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575753.591193:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788575753.591195:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575753.591199:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7666 00000100:00080000:3.0:1788575753.591203:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575753.591206:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575753.591211:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575753.591227:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2a00 x1875449616525824/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575753.591238:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575753.591274:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575753/real 1788575753] req@ffff9ada604e0e00 x1875449616525568/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575778 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575753.591291:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0e00 x1875449616525568/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575778 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575753.591300:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575753.591304:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575753.591308:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616525568:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575753.591328:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616525696:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575753.591349:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575753.591360:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575753/real 1788575753] req@ffff9ada604e1500 x1875449616525696/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575778 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575753.591372:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1500 x1875449616525696/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575778 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575753.591379:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575753.591382:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575753.591385:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616525696:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575753.591391:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616525824:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575753.591409:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575753.591419:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575753/real 1788575753] req@ffff9ada604e2a00 x1875449616525824/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575778 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575753.591430:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2a00 x1875449616525824/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575778 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575753.591436:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575753.591439:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575753.591442:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616525824:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788575753.782985:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575754.103536:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575754.806990:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575755.127048:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575755.830957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575756.151883:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575756.854989:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575757.175955:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575757.878934:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575758.198984:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575758.710978:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575758.710986:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575758.710989:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575758.711002:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7671 00000100:00080000:1.0:1788575758.711010:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575758.711016:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575758.711047:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e680 x1875449616525952/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575758.711066:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575758.711068:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575758.711074:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575758.711076:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7671 00000100:00080000:1.0:1788575758.711078:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575758.711083:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575758.711090:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575758.711098:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d880 x1875449616526080/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575758.711104:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575758.711107:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788575758.711108:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575758.711111:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7671 00000100:00080000:1.0:1788575758.711112:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575758.711115:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575758.711118:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575758.711123:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616525952:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788575758.711126:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ce00 x1875449616526208/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575758.711188:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575758.711220:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575758/real 1788575758] req@ffff9ada4566e680 x1875449616525952/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575783 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575758.711238:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e680 x1875449616525952/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575783 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575758.711248:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575758.711252:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575758.711255:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616525952:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575758.711262:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616526080:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575758.711283:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575758.711294:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575758/real 1788575758] req@ffff9ada4566d880 x1875449616526080/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575783 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575758.711305:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d880 x1875449616526080/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575783 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575758.711313:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575758.711316:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575758.711319:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616526080:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575758.711340:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616526208:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575758.711360:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575758.711369:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575758/real 1788575758] req@ffff9ada4566ce00 x1875449616526208/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575783 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575758.711379:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ce00 x1875449616526208/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575783 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575758.711385:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575758.711388:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575758.711391:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616526208:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575758.902952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575759.222998:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575759.926991:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575760.247034:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575760.950954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575761.270979:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575761.974968:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575762.295003:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575762.998947:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575763.319045:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575763.830933:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575763.830943:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575763.830946:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575763.830955:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7676 00000100:00080000:1.0:1788575763.830962:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575763.830968:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575763.830996:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c380 x1875449616526336/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575763.831019:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575763.831022:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575763.831024:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575763.831027:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7676 00000100:00080000:1.0:1788575763.831030:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575763.831033:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575763.831042:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575763.831055:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c700 x1875449616526464/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788575763.831069:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616526336:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788575763.831074:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575763.831079:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788575763.831081:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575763.831085:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7676 00000100:00080000:1.0:1788575763.831089:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575763.831093:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575763.831098:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575763.831111:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f100 x1875449616526592/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575763.831133:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575763.831162:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575763/real 1788575763] req@ffff9ada4566c380 x1875449616526336/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575763.831180:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c380 x1875449616526336/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575763.831190:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575763.831194:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575763.831198:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616526336:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575763.831215:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616526464:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575763.831235:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575763.831247:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575763/real 1788575763] req@ffff9ada4566c700 x1875449616526464/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575763.831260:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c700 x1875449616526464/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575763.831268:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575763.831271:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575763.831274:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616526464:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575763.831277:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616526592:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575763.831292:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575763.831299:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575763/real 1788575763] req@ffff9ada4566f100 x1875449616526592/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575788 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575763.831306:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f100 x1875449616526592/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575788 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575763.831310:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575763.831312:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575763.831314:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616526592:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575764.022934:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575764.343013:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575765.046944:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575765.366996:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575766.070934:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575766.391010:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575767.094939:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575767.415074:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575768.118946:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575768.438972:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788575768.950989:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575768.950999:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788575768.951004:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575768.951014:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7681 00000100:00080000:3.0:1788575768.951024:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575768.951030:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575768.951061:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3100 x1875449616526720/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575768.951080:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575768.951083:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788575768.951084:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575768.951086:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7681 00000100:00080000:3.0:1788575768.951088:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575768.951090:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575768.951097:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575768.951104:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2680 x1875449616526848/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575768.951110:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575768.951113:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788575768.951115:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575768.951118:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7681 00000100:00080000:3.0:1788575768.951120:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575768.951121:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575768.951124:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575768.951127:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616526720:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575768.951135:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1f80 x1875449616526976/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575768.951179:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575768.951228:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575768/real 1788575768] req@ffff9ada604e3100 x1875449616526720/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575768.951241:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3100 x1875449616526720/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575768.951248:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575768.951250:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575768.951253:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616526720:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575768.951258:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616526848:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575768.951273:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575768.951280:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575768/real 1788575768] req@ffff9ada604e2680 x1875449616526848/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575768.951287:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2680 x1875449616526848/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575768.951291:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575768.951293:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575768.951295:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616526848:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575768.951311:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616526976:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575768.951321:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575768.951326:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575768/real 1788575768] req@ffff9ada604e1f80 x1875449616526976/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575793 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575768.951332:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1f80 x1875449616526976/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575793 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575768.951337:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575768.951338:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575768.951340:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616526976:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575769.142955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575769.463018:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575770.166953:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575770.486958:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575771.190917:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575771.511003:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575772.214950:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575772.535045:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575773.238941:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575773.559057:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788575774.071000:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575774.071010:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788575774.071014:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575774.071025:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7686 00000100:00080000:3.0:1788575774.071033:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575774.071039:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575774.071071:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3b80 x1875449616527104/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575774.071106:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575774.071110:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788575774.071111:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575774.071116:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7686 00000100:00080000:3.0:1788575774.071119:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575774.071123:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575774.071143:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575774.071157:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1c00 x1875449616527232/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575774.071168:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1788575774.071169:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616527104:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788575774.071181:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788575774.071186:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575774.071191:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7686 00000100:00080000:3.0:1788575774.071195:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575774.071199:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575774.071204:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575774.071216:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0000 x1875449616527360/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575774.071240:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575774.071275:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575774/real 1788575774] req@ffff9ada604e3b80 x1875449616527104/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575799 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575774.071294:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3b80 x1875449616527104/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575799 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575774.071304:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575774.071307:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575774.071311:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616527104:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575774.071333:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616527232:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575774.071353:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575774.071362:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575774/real 1788575774] req@ffff9ada604e1c00 x1875449616527232/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575799 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575774.071370:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1c00 x1875449616527232/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575799 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575774.071388:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575774.071391:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575774.071394:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616527232:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575774.071403:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616527360:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575774.071424:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575774.071433:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575774/real 1788575774] req@ffff9ada604e0000 x1875449616527360/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575799 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575774.071441:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0000 x1875449616527360/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575799 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575774.071445:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575774.071447:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575774.071449:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616527360:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575774.262964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575774.583057:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575775.287008:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575775.607145:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575776.310988:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575776.631041:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575777.334943:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575777.655030:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575778.358899:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575778.679030:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575779.190964:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575779.190972:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575779.190976:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575779.190983:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7691 00000100:00080000:2.0:1788575779.190991:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575779.190997:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575779.191025:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616527488/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575779.191051:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575779.191054:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575779.191056:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575779.191059:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7691 00000100:00080000:2.0:1788575779.191063:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575779.191066:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575779.191075:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575779.191087:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616527616/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575779.191096:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575779.191099:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575779.191101:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1788575779.191106:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616527488:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788575779.191108:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7691 00000100:00080000:2.0:1788575779.191114:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575779.191118:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575779.191124:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575779.191135:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616527744/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788575779.191173:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616527616:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575779.191198:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616527744:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575779.382980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575779.703049:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575780.406994:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575780.727048:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575781.430993:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575781.751098:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1788575782.263121:0:2370:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:2.0:1788575782.263158:0:2370:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1788575782.263173:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788575782.263213:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788575782.263223:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:0.0:1788575782.263307:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575779/real 1788575782] req@ffff9ada60624380 x1875449616527744/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575804 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575782.263345:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616527744/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575804 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575782.263358:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575782.263364:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575782.263368:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616527744:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575782.263377:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575779/real 1788575782] req@ffff9ada60625880 x1875449616527488/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575804 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575782.263391:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616527488/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575804 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575782.263399:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575782.263402:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575782.263406:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616527488:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575782.263413:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575779/real 1788575782] req@ffff9ada60624a80 x1875449616527616/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575804 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575782.263426:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616527616/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575804 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575782.263434:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575782.263438:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575782.263441:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616527616:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575782.454997:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575782.774978:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575783.478930:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575783.799018:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575784.311018:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575784.311026:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575784.311029:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575784.311037:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7696 00000100:00080000:2.0:1788575784.311046:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575784.311052:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575784.311087:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616527872/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575784.311116:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575784.311119:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575784.311121:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575784.311124:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7696 00000100:00080000:2.0:1788575784.311128:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575784.311132:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575784.311141:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575784.311155:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616528000/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575784.311166:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575784.311169:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:0.0:1788575784.311172:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616527872:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788575784.311172:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575784.311179:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7696 00000100:00080000:2.0:1788575784.311184:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575784.311186:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575784.311190:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575784.311200:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616528128/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575784.311252:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575784.311280:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575784/real 1788575784] req@ffff9ada60625500 x1875449616527872/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575809 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575784.311294:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616527872/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575809 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575784.311302:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575784.311305:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575784.311307:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616527872:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575784.311329:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616528000:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575784.311353:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575784.311364:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575784/real 1788575784] req@ffff9ada60626a00 x1875449616528000/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575809 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575784.311375:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616528000/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575809 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575784.311382:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575784.311385:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575784.311388:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616528000:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575784.311393:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616528128:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575784.311411:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575784.311421:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575784/real 1788575784] req@ffff9ada60626680 x1875449616528128/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575809 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575784.311431:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616528128/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575809 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575784.311438:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575784.311441:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575784.311443:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616528128:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575784.502963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575784.823054:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575785.526930:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575785.847006:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575786.550970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575786.871015:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575787.574951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575787.895031:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575788.598960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575788.919005:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575789.430972:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575789.430981:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575789.430985:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575789.430995:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7701 00000100:00080000:0.0:1788575789.431003:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575789.431010:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575789.431042:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616528256/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575789.431069:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575789.431072:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575789.431074:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575789.431077:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7701 00000100:00080000:0.0:1788575789.431080:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575789.431091:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575789.431103:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575789.431114:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616528384/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575789.431123:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616528256:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788575789.431126:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575789.431132:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575789.431134:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575789.431138:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7701 00000100:00080000:0.0:1788575789.431141:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575789.431145:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575789.431149:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575789.431159:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616528512/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575789.431186:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575789.431221:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575789/real 1788575789] req@ffff9ada4557e300 x1875449616528256/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575814 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575789.431240:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616528256/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575814 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575789.431251:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575789.431255:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575789.431260:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616528256:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575789.431281:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616528384:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575789.431301:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575789.431313:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575789/real 1788575789] req@ffff9ada4557fb80 x1875449616528384/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575814 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575789.431325:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616528384/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575814 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575789.431334:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575789.431338:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575789.431341:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616528384:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575789.431347:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616528512:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575789.431363:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575789.431372:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575789/real 1788575789] req@ffff9ada4557ed80 x1875449616528512/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575814 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575789.431396:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616528512/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575814 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575789.431403:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575789.431406:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575789.431409:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616528512:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575789.622960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575789.942967:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575790.646955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575790.967041:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575791.670939:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575791.991061:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575792.694927:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575793.015044:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575793.718933:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575794.038975:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575794.550973:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575794.550981:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575794.550985:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575794.550993:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7707 00000100:00080000:1.0:1788575794.551002:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575794.551008:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575794.551039:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d180 x1875449616528640/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575794.551062:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575794.551064:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575794.551065:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575794.551069:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7707 00000100:00080000:1.0:1788575794.551072:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575794.551075:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575794.551083:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575794.551093:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616528640:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788575794.551094:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ca80 x1875449616528768/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575794.551106:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575794.551108:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788575794.551109:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575794.551112:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7707 00000100:00080000:1.0:1788575794.551114:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575794.551116:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575794.551118:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575794.551126:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d500 x1875449616528896/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575794.551148:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575794.551173:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575794/real 1788575794] req@ffff9ada4566d180 x1875449616528640/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575819 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575794.551185:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d180 x1875449616528640/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575819 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575794.551192:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575794.551194:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575794.551197:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616528640:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575794.551212:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616528768:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575794.551224:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575794.551244:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575794/real 1788575794] req@ffff9ada4566ca80 x1875449616528768/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575819 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575794.551257:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ca80 x1875449616528768/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575819 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575794.551265:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575794.551268:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575794.551272:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616528768:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575794.551278:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616528896:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575794.551293:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575794.551301:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575794/real 1788575794] req@ffff9ada4566d500 x1875449616528896/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575819 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575794.551307:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d500 x1875449616528896/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575819 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575794.551312:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575794.551314:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575794.551316:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616528896:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575794.743013:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575795.063014:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575795.766964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575796.087118:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575796.790956:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575797.111052:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575797.814954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575798.135004:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575798.838949:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575799.159010:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575799.671691:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575799.671700:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575799.671705:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575799.671715:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7712 00000100:00080000:0.0:1788575799.671724:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575799.671731:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575799.671766:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616529024/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575799.671798:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575799.671802:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575799.671815:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575799.671819:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7712 00000100:00080000:0.0:1788575799.671823:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575799.671827:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575799.671836:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575799.671888:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d180 x1875449616529152/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575799.671890:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616529024:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788575799.671908:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575799.671912:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575799.671914:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575799.671919:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7712 00000100:00080000:0.0:1788575799.671923:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575799.671927:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575799.671932:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575799.671944:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616529280/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575799.671955:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575799.671989:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575799/real 1788575799] req@ffff9ada4557e680 x1875449616529024/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575824 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575799.672007:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616529024/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575824 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575799.672017:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575799.672021:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575799.672024:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616529024:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575799.672044:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616529152:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575799.672062:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575799.672072:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575799/real 1788575799] req@ffff9ada4557d180 x1875449616529152/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575824 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575799.672083:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d180 x1875449616529152/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575824 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575799.672105:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575799.672108:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575799.672112:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616529152:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575799.672117:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616529280:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575799.672136:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575799.672146:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575799/real 1788575799] req@ffff9ada4557ea00 x1875449616529280/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575824 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575799.672156:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616529280/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575824 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575799.672163:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575799.672166:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575799.672168:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616529280:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575799.862954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575800.182988:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575800.886967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575801.207043:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575801.910955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575802.231040:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575802.934943:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575803.255044:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575803.958981:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575804.279049:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575804.790982:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575804.790992:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575804.790997:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575804.791007:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7717 00000100:00080000:2.0:1788575804.791016:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575804.791025:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575804.791062:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627100 x1875449616529408/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575804.791092:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575804.791097:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575804.791100:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575804.791105:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7717 00000100:00080000:2.0:1788575804.791109:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575804.791114:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575804.791123:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575804.791127:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616529408:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788575804.791140:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624700 x1875449616529536/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575804.791154:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575804.791157:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575804.791159:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575804.791168:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7717 00000100:00080000:2.0:1788575804.791190:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575804.791195:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575804.791200:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1788575804.791202:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788575804.791220:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616529664/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575804.791258:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575804/real 1788575804] req@ffff9ada60627100 x1875449616529408/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575829 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575804.791283:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627100 x1875449616529408/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575829 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575804.791295:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575804.791300:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575804.791304:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616529408:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575804.791328:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616529536:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575804.791347:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575804.791356:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575804/real 1788575804] req@ffff9ada60624700 x1875449616529536/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575829 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575804.791364:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624700 x1875449616529536/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575829 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575804.791369:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575804.791372:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575804.791374:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616529536:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575804.791377:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616529664:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575804.791387:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575804.791393:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575804/real 1788575804] req@ffff9ada60626680 x1875449616529664/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575829 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575804.791399:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616529664/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575829 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575804.791404:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575804.791406:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575804.791407:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616529664:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788575804.983032:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575805.303022:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575806.006982:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575806.327096:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575807.031004:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575807.351145:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575808.054990:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575808.375047:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575809.078982:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575809.399121:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575809.911008:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575809.911022:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575809.911029:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575809.911043:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7722 00000100:00080000:2.0:1788575809.911056:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575809.911065:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575809.911117:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616529792/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575809.911152:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575809.911156:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575809.911158:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575809.911162:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7722 00000100:00080000:2.0:1788575809.911166:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575809.911170:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575809.911182:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575809.911193:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616529920/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575809.911200:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575809.911204:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575809.911206:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1788575809.911207:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616529792:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788575809.911212:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7722 00000100:00080000:2.0:1788575809.911217:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575809.911219:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575809.911222:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575809.911229:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616530048/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575809.911299:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575809.911341:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575809/real 1788575809] req@ffff9ada60626a00 x1875449616529792/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575834 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575809.911364:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616529792/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575834 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575809.911376:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575809.911381:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575809.911384:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616529792:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575809.911412:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616529920:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575809.911436:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575809.911449:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575809/real 1788575809] req@ffff9ada60625500 x1875449616529920/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575834 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575809.911471:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616529920/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575834 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575809.911477:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575809.911479:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575809.911481:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616529920:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575809.911486:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616530048:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575809.911509:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575809.911520:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575809/real 1788575809] req@ffff9ada60624a80 x1875449616530048/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575834 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575809.911546:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616530048/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575834 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575809.911555:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575809.911558:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575809.911561:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616530048:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788575810.102970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575810.423148:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575811.126951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575811.447028:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575812.150949:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575812.471043:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575813.174984:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575813.495028:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575814.198964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575814.519052:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788575815.031009:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575815.031018:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788575815.031023:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575815.031033:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7727 00000100:00080000:3.0:1788575815.031045:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575815.031052:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575815.031096:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2300 x1875449616530176/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575815.031128:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575815.031132:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788575815.031134:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575815.031138:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7727 00000100:00080000:3.0:1788575815.031141:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575815.031145:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575815.031156:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575815.031171:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0a80 x1875449616530304/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575815.031181:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575815.031184:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788575815.031186:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575815.031190:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7727 00000100:00080000:3.0:1788575815.031194:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575815.031198:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575815.031204:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575815.031215:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616530176:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575815.031220:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2d80 x1875449616530432/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575815.031306:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575815.031342:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575815/real 1788575815] req@ffff9ada604e2300 x1875449616530176/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575840 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575815.031359:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2300 x1875449616530176/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575840 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575815.031375:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575815.031379:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575815.031383:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616530176:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575815.031390:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616530304:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575815.031409:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575815.031420:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575815/real 1788575815] req@ffff9ada604e0a80 x1875449616530304/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575840 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575815.031431:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0a80 x1875449616530304/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575840 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575815.031438:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575815.031441:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575815.031444:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616530304:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575815.031466:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616530432:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575815.031483:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575815.031492:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575815/real 1788575815] req@ffff9ada604e2d80 x1875449616530432/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575840 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575815.031501:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2d80 x1875449616530432/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575840 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575815.031508:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575815.031511:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575815.031514:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616530432:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788575815.223003:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575815.543036:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575816.246992:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575816.567081:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575817.270980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575817.591057:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575818.294937:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575818.615048:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575819.318951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575819.639011:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575820.150992:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575820.151002:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575820.151006:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575820.151014:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7732 00000100:00080000:2.0:1788575820.151024:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575820.151031:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575820.151068:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616530560/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575820.151101:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575820.151105:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575820.151107:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575820.151111:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7732 00000100:00080000:2.0:1788575820.151114:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575820.151118:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575820.151127:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575820.151142:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616530560:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788575820.151143:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616530688/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575820.151159:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575820.151161:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575820.151163:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575820.151167:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7732 00000100:00080000:2.0:1788575820.151171:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575820.151175:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575820.151179:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575820.151190:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616530816/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575820.151206:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575820.151250:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575820/real 1788575820] req@ffff9ada60625880 x1875449616530560/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575845 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575820.151273:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616530560/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575845 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575820.151284:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575820.151288:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575820.151291:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616530560:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575820.151320:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616530688:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575820.151342:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575820.151355:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575820/real 1788575820] req@ffff9ada60624380 x1875449616530688/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575845 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575820.151375:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616530688/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575845 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575820.151383:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575820.151386:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575820.151389:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616530688:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575820.151395:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616530816:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575820.151413:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575820.151423:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575820/real 1788575820] req@ffff9ada60625c00 x1875449616530816/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575845 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575820.151435:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616530816/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575845 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575820.151442:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575820.151445:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575820.151448:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616530816:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788575820.342927:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575820.663039:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575821.367002:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575821.687134:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575822.390930:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575822.711016:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575823.414993:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575823.735190:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575824.438994:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575824.759027:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575825.270992:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575825.271000:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575825.271011:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575825.271023:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7737 00000100:00080000:1.0:1788575825.271034:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575825.271041:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575825.271077:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ea00 x1875449616530944/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575825.271109:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575825.271113:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575825.271114:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575825.271117:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7737 00000100:00080000:1.0:1788575825.271120:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575825.271122:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575825.271130:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575825.271140:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f480 x1875449616531072/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575825.271146:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575825.271148:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788575825.271149:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575825.271151:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7737 00000100:00080000:1.0:1788575825.271153:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575825.271155:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575825.271157:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575825.271161:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e300 x1875449616531200/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788575825.271236:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616530944:192.168.202.113@tcp:250:kworker.0 00000800:00000100:0.0:1788575825.271297:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575825.271329:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575825/real 1788575825] req@ffff9ada4566ea00 x1875449616530944/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575850 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575825.271345:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ea00 x1875449616530944/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575850 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575825.271353:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575825.271355:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575825.271358:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616530944:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575825.271363:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616531072:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575825.271397:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575825.271410:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575825/real 1788575825] req@ffff9ada4566f480 x1875449616531072/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575850 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575825.271420:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f480 x1875449616531072/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575850 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575825.271433:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575825.271437:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575825.271440:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616531072:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575825.271446:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616531200:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575825.271462:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575825.271471:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575825/real 1788575825] req@ffff9ada4566e300 x1875449616531200/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575850 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575825.271481:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e300 x1875449616531200/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575850 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575825.271488:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575825.271491:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575825.271494:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616531200:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788575825.462966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575825.783032:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575826.487547:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575826.806989:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575827.510957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575827.831093:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575828.534993:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575828.855007:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575829.558936:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575829.879067:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788575830.391017:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575830.391031:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788575830.391036:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575830.391050:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7742 00000100:00080000:3.0:1788575830.391062:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575830.391071:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575830.391116:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1880 x1875449616531328/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575830.391144:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575830.391147:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788575830.391149:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575830.391151:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7742 00000100:00080000:3.0:1788575830.391154:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575830.391156:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575830.391166:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575830.391175:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3800 x1875449616531456/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575830.391182:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575830.391184:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788575830.391186:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575830.391188:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7742 00000100:00080000:3.0:1788575830.391189:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575830.391191:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575830.391194:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575830.391203:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1180 x1875449616531584/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788575830.391212:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616531328:192.168.202.113@tcp:250:kworker.0 00000800:00000100:0.0:1788575830.391318:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575830.391360:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575830/real 1788575830] req@ffff9ada604e1880 x1875449616531328/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575855 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575830.391380:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1880 x1875449616531328/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575855 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575830.391392:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575830.391397:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575830.391401:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616531328:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575830.391408:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616531456:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575830.391431:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575830.391442:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575830/real 1788575830] req@ffff9ada604e3800 x1875449616531456/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575855 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575830.391454:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3800 x1875449616531456/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575855 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575830.391462:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575830.391466:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575830.391469:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616531456:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575830.391495:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616531584:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575830.391511:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575830.391519:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575830/real 1788575830] req@ffff9ada604e1180 x1875449616531584/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575855 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575830.391530:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1180 x1875449616531584/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575855 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575830.391537:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575830.391540:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575830.391544:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616531584:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788575830.583022:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575830.903068:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575831.606990:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575831.927063:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575832.631012:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575832.951031:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575833.655011:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575833.975057:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575834.678990:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575834.999029:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788575835.510944:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575835.510957:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788575835.510962:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575835.510974:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7748 00000100:00080000:3.0:1788575835.510988:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575835.510996:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575835.511039:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0700 x1875449616531712/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575835.511073:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575835.511076:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788575835.511078:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575835.511088:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7748 00000100:00080000:3.0:1788575835.511091:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575835.511107:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575835.511118:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575835.511131:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3480 x1875449616531840/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575835.511141:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575835.511146:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:0.0:1788575835.511148:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616531712:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788575835.511149:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575835.511158:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7748 00000100:00080000:3.0:1788575835.511161:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575835.511164:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575835.511168:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575835.511179:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0380 x1875449616531968/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575835.511250:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575835.511297:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575835/real 1788575835] req@ffff9ada604e0700 x1875449616531712/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575860 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575835.511313:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0700 x1875449616531712/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575860 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575835.511322:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575835.511325:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575835.511328:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616531712:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575835.511349:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616531840:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575835.511367:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575835.511378:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575835/real 1788575835] req@ffff9ada604e3480 x1875449616531840/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575860 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575835.511390:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3480 x1875449616531840/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575860 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575835.511398:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575835.511402:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575835.511405:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616531840:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575835.511411:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616531968:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575835.511428:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575835.511436:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575835/real 1788575835] req@ffff9ada604e0380 x1875449616531968/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575860 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575835.511443:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0380 x1875449616531968/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575860 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575835.511449:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575835.511451:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575835.511453:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616531968:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788575835.703305:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575836.023294:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575836.726978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575837.046983:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575837.750971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575838.071012:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575838.775030:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575839.095185:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575839.798941:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575840.119048:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575840.631001:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575840.631010:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575840.631015:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575840.631024:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7753 00000100:00080000:2.0:1788575840.631033:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575840.631040:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575840.631075:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616532096/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575840.631117:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575840.631124:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575840.631126:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575840.631130:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7753 00000100:00080000:2.0:1788575840.631134:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575840.631137:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575840.631145:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575840.631159:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616532224/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788575840.631165:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616532096:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788575840.631178:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575840.631182:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575840.631184:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575840.631189:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7753 00000100:00080000:2.0:1788575840.631193:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575840.631198:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575840.631204:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575840.631215:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616532352/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575840.631223:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575840.631254:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575840/real 1788575840] req@ffff9ada60624e00 x1875449616532096/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575865 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575840.631276:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616532096/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575865 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575840.631284:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575840.631286:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575840.631288:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616532096:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575840.631305:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616532224:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575840.631319:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575840.631326:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575840/real 1788575840] req@ffff9ada60627480 x1875449616532224/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575865 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575840.631334:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616532224/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575865 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575840.631339:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575840.631341:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575840.631343:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616532224:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575840.631346:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616532352:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575840.631355:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575840.631360:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575840/real 1788575840] req@ffff9ada60625f80 x1875449616532352/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575865 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575840.631366:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616532352/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575865 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575840.631370:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575840.631372:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575840.631373:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616532352:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788575840.823091:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575841.143057:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575841.846988:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575842.167125:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575842.870979:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575843.191786:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575843.894978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575844.215717:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575844.919153:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575845.239076:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575845.750990:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575845.751000:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575845.751005:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575845.751014:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7758 00000100:00080000:0.0:1788575845.751022:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575845.751030:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575845.751064:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557df80 x1875449616532480/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575845.751093:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575845.751098:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575845.751099:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575845.751103:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7758 00000100:00080000:0.0:1788575845.751106:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575845.751109:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575845.751117:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575845.751129:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d500 x1875449616532608/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575845.751138:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575845.751141:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575845.751143:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575845.751146:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7758 00000100:00080000:0.0:1788575845.751151:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1788575845.751152:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616532480:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788575845.751157:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575845.751164:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575845.751175:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616532736/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575845.751239:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616532608:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575845.751277:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616532736:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788575845.942979:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575846.263047:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575846.966964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575847.287196:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575847.990980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575848.311058:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:0.0:1788575848.823137:0:2372:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:0.0:1788575848.823185:0:2372:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:0.0:1788575848.823205:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:0.0:1788575848.823254:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:0.0:1788575848.823263:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:3.0:1788575848.823591:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575845/real 1788575848] req@ffff9ada4557ea00 x1875449616532736/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575870 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575848.823626:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616532736/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575870 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575848.823636:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575848.823641:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575848.823645:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616532736:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575848.823652:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575845/real 1788575848] req@ffff9ada4557df80 x1875449616532480/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575870 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575848.823663:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557df80 x1875449616532480/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575870 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575848.823669:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575848.823672:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575848.823675:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616532480:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575848.823681:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575845/real 1788575848] req@ffff9ada4557d500 x1875449616532608/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575870 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575848.823696:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d500 x1875449616532608/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575870 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575848.823704:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575848.823707:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575848.823711:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616532608:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788575849.015093:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575849.335376:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575850.038981:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575850.358988:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788575850.870977:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575850.870985:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788575850.870988:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575850.870995:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7763 00000100:00080000:3.0:1788575850.871003:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575850.871009:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575850.871036:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0380 x1875449616532864/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575850.871057:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575850.871059:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788575850.871060:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575850.871063:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7763 00000100:00080000:3.0:1788575850.871065:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575850.871067:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575850.871074:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575850.871091:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616532864:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575850.871097:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3480 x1875449616532992/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575850.871109:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575850.871113:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788575850.871115:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575850.871119:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7763 00000100:00080000:3.0:1788575850.871123:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575850.871127:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575850.871132:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575850.871150:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0700 x1875449616533120/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575850.871157:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575850.871193:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575850/real 1788575850] req@ffff9ada604e0380 x1875449616532864/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575875 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575850.871214:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0380 x1875449616532864/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575875 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575850.871225:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575850.871229:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575850.871232:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616532864:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575850.871258:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616532992:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575850.871280:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575850.871302:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575850/real 1788575850] req@ffff9ada604e3480 x1875449616532992/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575875 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575850.871319:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3480 x1875449616532992/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575875 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575850.871327:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575850.871330:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575850.871334:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616532992:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575850.871340:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616533120:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575850.871360:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575850.871379:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575850/real 1788575850] req@ffff9ada604e0700 x1875449616533120/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575875 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575850.871391:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0700 x1875449616533120/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575875 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575850.871398:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575850.871401:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575850.871404:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616533120:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788575851.062968:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575851.383026:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575852.086971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575852.407040:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575853.111100:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575853.431192:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575854.134964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575854.455019:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575855.158955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575855.479025:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788575855.990978:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575855.990988:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788575855.990993:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575855.991002:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7768 00000100:00080000:3.0:1788575855.991012:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575855.991020:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575855.991055:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1180 x1875449616533248/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575855.991083:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575855.991086:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788575855.991088:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788575855.991092:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7768 00000100:00080000:3.0:1788575855.991094:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575855.991098:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575855.991107:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575855.991117:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3800 x1875449616533376/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788575855.991128:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788575855.991131:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788575855.991133:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1788575855.991137:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616533248:192.168.202.113@tcp:250:kworker.0 00000100:00080000:3.0:1788575855.991139:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7768 00000100:00080000:3.0:1788575855.991145:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788575855.991149:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788575855.991153:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575855.991162:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1880 x1875449616533504/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1788575855.991207:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575855.991237:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575855/real 1788575855] req@ffff9ada604e1180 x1875449616533248/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575880 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575855.991252:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1180 x1875449616533248/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575880 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575855.991262:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575855.991266:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575855.991270:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616533248:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575855.991292:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616533376:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575855.991314:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575855.991324:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575855/real 1788575855] req@ffff9ada604e3800 x1875449616533376/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575880 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575855.991335:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3800 x1875449616533376/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575880 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575855.991342:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575855.991344:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575855.991347:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616533376:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575855.991351:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616533504:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575855.991366:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575855.991374:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575855/real 1788575855] req@ffff9ada604e1880 x1875449616533504/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575880 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575855.991383:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1880 x1875449616533504/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575880 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575855.991389:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575855.991392:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575855.991394:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616533504:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788575856.182989:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575856.503003:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575857.206946:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575857.527059:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575858.230978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575858.550994:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575859.254920:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575859.575025:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575860.278991:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575860.599084:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575861.110990:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575861.111004:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575861.111009:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575861.111033:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7773 00000100:00080000:2.0:1788575861.111047:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575861.111056:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575861.111109:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616533632/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575861.111137:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575861.111140:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575861.111141:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575861.111144:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7773 00000100:00080000:2.0:1788575861.111147:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575861.111150:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575861.111160:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575861.111170:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616533760/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575861.111178:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575861.111181:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575861.111183:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575861.111187:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7773 00000100:00080000:2.0:1788575861.111190:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575861.111193:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575861.111198:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575861.111208:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616533888/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788575861.111355:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616533632:192.168.202.113@tcp:250:kworker.0 00000800:00000100:0.0:1788575861.111426:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575861.111462:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575861/real 1788575861] req@ffff9ada60625180 x1875449616533632/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575886 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575861.111479:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616533632/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575886 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575861.111491:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575861.111507:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575861.111511:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616533632:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788575861.111519:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616533760:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575861.111541:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575861.111552:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575861/real 1788575861] req@ffff9ada60626d80 x1875449616533760/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575886 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575861.111564:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616533760/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575886 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575861.111572:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575861.111575:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575861.111578:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616533760:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788575861.111584:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616533888:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788575861.111601:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788575861.111610:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575861/real 1788575861] req@ffff9ada60627b80 x1875449616533888/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575886 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788575861.111626:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616533888/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575886 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788575861.111632:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788575861.111635:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788575861.111639:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616533888:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788575861.303022:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575861.623142:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575862.326946:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575862.647057:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575863.350983:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575863.673530:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575864.375111:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575864.695124:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575865.398978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575865.722465:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575866.231000:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575866.231012:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575866.231017:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575866.231029:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7778 00000100:00080000:0.0:1788575866.231042:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575866.231050:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575866.231093:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d180 x1875449616534016/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575866.231128:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575866.231132:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575866.231134:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575866.231138:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7778 00000100:00080000:0.0:1788575866.231141:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575866.231144:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575866.231154:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575866.231166:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616534144/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575866.231176:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575866.231180:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575866.231181:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575866.231185:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7778 00000100:00080000:0.0:1788575866.231189:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1788575866.231190:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616534016:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788575866.231195:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575866.231201:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575866.231210:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616534272/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575866.231268:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575866.231309:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575866/real 1788575866] req@ffff9ada4557d180 x1875449616534016/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575891 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575866.231329:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d180 x1875449616534016/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575891 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575866.231340:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575866.231345:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575866.231349:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616534016:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575866.231356:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616534144:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575866.231378:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575866.231388:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575866/real 1788575866] req@ffff9ada4557e680 x1875449616534144/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575891 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575866.231426:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616534144/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575891 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575866.231434:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575866.231437:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575866.231441:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616534144:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575866.231466:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616534272:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575866.231497:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575866.231508:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575866/real 1788575866] req@ffff9ada4557ed80 x1875449616534272/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575891 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575866.231518:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616534272/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575891 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575866.231526:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575866.231529:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575866.231532:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616534272:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788575866.422936:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575866.743044:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575867.446977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575867.767409:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575868.471000:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575868.793600:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575869.495130:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575869.815010:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575870.518992:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575870.839068:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575871.351041:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575871.351050:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575871.351055:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575871.351064:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7783 00000100:00080000:2.0:1788575871.351074:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575871.351080:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575871.351128:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627800 x1875449616534400/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575871.351154:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575871.351157:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575871.351159:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575871.351163:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7783 00000100:00080000:2.0:1788575871.351166:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575871.351170:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575871.351179:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575871.351190:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626300 x1875449616534528/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575871.351200:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575871.351204:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575871.351205:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575871.351209:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7783 00000100:00080000:2.0:1788575871.351211:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575871.351217:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575871.351217:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616534400:192.168.202.113@tcp:250:kworker.0 00000008:00100000:2.0:1788575871.351222:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575871.351235:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616534656/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575871.351276:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575871.351321:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575871/real 1788575871] req@ffff9ada60627800 x1875449616534400/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575896 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575871.351339:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627800 x1875449616534400/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575896 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575871.351349:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575871.351353:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575871.351356:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616534400:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575871.351376:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616534528:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575871.351404:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575871.351415:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575871/real 1788575871] req@ffff9ada60626300 x1875449616534528/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575896 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575871.351426:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626300 x1875449616534528/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575896 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575871.351434:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575871.351437:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575871.351440:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616534528:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575871.351446:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616534656:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575871.351462:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575871.351472:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575871/real 1788575871] req@ffff9ada60627b80 x1875449616534656/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575896 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575871.351495:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616534656/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575896 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575871.351502:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575871.351505:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575871.351508:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616534656:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788575871.543030:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575871.863074:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575872.567017:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575872.887091:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575873.591004:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575873.911089:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575874.615138:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575874.935359:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575875.638975:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575875.959267:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575876.471040:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575876.471050:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575876.471054:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575876.471063:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7789 00000100:00080000:1.0:1788575876.471072:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575876.471078:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575876.471113:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f800 x1875449616534784/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575876.471147:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575876.471151:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575876.471153:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575876.471157:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7789 00000100:00080000:1.0:1788575876.471160:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575876.471164:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575876.471172:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575876.471196:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566dc00 x1875449616534912/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575876.471206:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575876.471209:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788575876.471210:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575876.471214:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7789 00000100:00080000:1.0:1788575876.471217:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575876.471221:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575876.471226:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575876.471234:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ed80 x1875449616535040/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575876.471330:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616534784:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788575876.471380:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575876.471408:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575876/real 1788575876] req@ffff9ada4566f800 x1875449616534784/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575901 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575876.471425:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f800 x1875449616534784/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575901 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575876.471435:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575876.471439:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575876.471442:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616534784:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575876.471449:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616534912:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575876.471466:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575876.471477:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575876/real 1788575876] req@ffff9ada4566dc00 x1875449616534912/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575901 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575876.471496:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566dc00 x1875449616534912/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575901 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575876.471504:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575876.471507:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575876.471510:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616534912:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575876.471515:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616535040:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575876.471531:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575876.471540:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575876/real 1788575876] req@ffff9ada4566ed80 x1875449616535040/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575901 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575876.471551:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ed80 x1875449616535040/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575901 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575876.471558:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575876.471561:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575876.471564:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616535040:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788575876.662970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575876.983052:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788575877.686981:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575878.007213:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575878.710946:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575879.031073:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575879.734963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575880.055763:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575880.758999:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575881.079068:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575881.591660:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575881.591667:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575881.591670:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575881.591678:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7794 00000100:00080000:1.0:1788575881.591687:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575881.591692:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575881.591721:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c000 x1875449616535168/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575881.591752:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575881.591756:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575881.591757:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575881.591761:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7794 00000100:00080000:1.0:1788575881.591764:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575881.591767:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575881.591776:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575881.591788:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566df80 x1875449616535296/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575881.591797:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575881.591799:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788575881.591801:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575881.591804:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7794 00000100:00080000:1.0:1788575881.591807:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575881.591809:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575881.591813:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575881.591821:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566fb80 x1875449616535424/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575881.591850:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616535168:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788575881.591933:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575881.591963:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575881/real 1788575881] req@ffff9ada4566c000 x1875449616535168/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575906 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575881.591980:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c000 x1875449616535168/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575906 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575881.591990:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575881.591994:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575881.591998:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616535168:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575881.592004:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616535296:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575881.592026:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575881.592038:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575881/real 1788575881] req@ffff9ada4566df80 x1875449616535296/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575906 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575881.592050:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566df80 x1875449616535296/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575906 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575881.592058:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575881.592061:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575881.592064:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616535296:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575881.592070:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616535424:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575881.592100:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575881.592108:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575881/real 1788575881] req@ffff9ada4566fb80 x1875449616535424/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575906 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575881.592120:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566fb80 x1875449616535424/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575906 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575881.592125:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575881.592127:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575881.592129:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616535424:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575881.782975:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575882.103082:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575882.806961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575883.127031:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575883.830996:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575884.151093:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575884.854931:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575885.175049:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575885.879140:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575886.199081:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575886.711011:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575886.711020:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575886.711025:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575886.711036:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7799 00000100:00080000:2.0:1788575886.711045:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575886.711052:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575886.711089:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616535552/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575886.711118:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575886.711122:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575886.711125:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575886.711129:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7799 00000100:00080000:2.0:1788575886.711132:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575886.711136:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575886.711145:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575886.711158:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616535680/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575886.711163:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616535552:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788575886.711176:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575886.711180:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575886.711182:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575886.711187:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7799 00000100:00080000:2.0:1788575886.711190:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575886.711194:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575886.711199:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575886.711211:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616535808/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575886.711218:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575886.711248:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575886/real 1788575886] req@ffff9ada60626d80 x1875449616535552/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575911 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575886.711261:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616535552/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575911 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575886.711268:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575886.711270:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575886.711274:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616535552:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575886.711308:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616535680:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575886.711335:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575886.711347:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575886/real 1788575886] req@ffff9ada60625180 x1875449616535680/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575911 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575886.711359:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616535680/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575911 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575886.711366:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575886.711369:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575886.711372:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616535680:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575886.711377:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616535808:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575886.711396:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575886.711407:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575886/real 1788575886] req@ffff9ada60625f80 x1875449616535808/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575911 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575886.711419:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616535808/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575911 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575886.711426:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575886.711430:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575886.711433:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616535808:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575886.902978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575887.223058:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575887.926978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575888.248009:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575888.950978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575889.271027:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575889.974981:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575890.295139:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575890.998974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575891.320097:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575891.831056:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575891.831067:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575891.831071:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575891.831081:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7804 00000100:00080000:1.0:1788575891.831091:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575891.831098:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575891.831132:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566fb80 x1875449616535936/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575891.831155:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575891.831157:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575891.831159:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575891.831161:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7804 00000100:00080000:1.0:1788575891.831164:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575891.831166:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575891.831174:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575891.831197:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566df80 x1875449616536064/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575891.831212:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616535936:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788575891.831218:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575891.831222:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788575891.831224:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575891.831229:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7804 00000100:00080000:1.0:1788575891.831233:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575891.831237:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575891.831242:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575891.831254:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c000 x1875449616536192/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575891.831277:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575891.831310:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575891/real 1788575891] req@ffff9ada4566fb80 x1875449616535936/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575916 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575891.831328:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566fb80 x1875449616535936/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575916 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575891.831339:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575891.831343:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575891.831346:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616535936:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575891.831368:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616536064:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575891.831389:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575891.831401:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575891/real 1788575891] req@ffff9ada4566df80 x1875449616536064/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575916 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575891.831413:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566df80 x1875449616536064/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575916 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575891.831421:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575891.831424:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575891.831427:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616536064:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575891.831433:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616536192:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575891.831448:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575891.831459:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575891/real 1788575891] req@ffff9ada4566c000 x1875449616536192/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575916 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575891.831470:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c000 x1875449616536192/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575916 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575891.831478:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575891.831481:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575891.831484:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616536192:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575892.022981:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575892.343366:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575893.047976:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575893.367036:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575894.070954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575894.391038:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575895.094961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575895.415104:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575896.118958:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575896.439121:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575896.951176:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575896.951187:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575896.951191:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575896.951201:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7809 00000100:00080000:0.0:1788575896.951210:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575896.951217:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575896.951262:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616536320/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575896.951293:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575896.951296:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575896.951298:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575896.951301:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7809 00000100:00080000:0.0:1788575896.951305:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575896.951308:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575896.951316:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575896.951334:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616536448/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575896.951344:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575896.951346:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575896.951348:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575896.951351:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7809 00000100:00080000:0.0:1788575896.951354:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575896.951358:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575896.951362:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575896.951371:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616536576/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575896.952657:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616536320:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788575896.952716:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575896.952748:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575896/real 1788575896] req@ffff9ada4557fb80 x1875449616536320/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575921 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575896.952793:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616536320/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575921 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575896.952803:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575896.952807:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575896.952811:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616536320:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575896.952818:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616536448:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575896.952840:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575896.952873:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575896/real 1788575896] req@ffff9ada4557e300 x1875449616536448/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575921 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575896.952886:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616536448/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575921 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575896.952894:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575896.952897:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575896.952900:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616536448:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575896.952906:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616536576:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575896.952925:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575896.952935:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575896/real 1788575896] req@ffff9ada4557f800 x1875449616536576/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575921 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575896.952946:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616536576/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575921 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575896.952953:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575896.952956:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575896.952959:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616536576:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575897.142966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575897.463040:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575898.166964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575898.487050:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575899.190975:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575899.511189:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575900.214981:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575900.535249:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575901.239018:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575901.559151:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575902.071118:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575902.071132:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575902.071137:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575902.071148:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7814 00000100:00080000:2.0:1788575902.071160:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575902.071168:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575902.071216:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616536704/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575902.071250:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575902.071254:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575902.071255:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575902.071259:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7814 00000100:00080000:2.0:1788575902.071263:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575902.071267:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575902.071292:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616536704:192.168.202.113@tcp:250:kworker.0 00000008:00100000:2.0:1788575902.071375:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000800:00000100:3.0:1788575902.071387:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788575902.071392:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616536832/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575902.071404:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575902.071409:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575902.071410:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575902.071414:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7814 00000100:00080000:2.0:1788575902.071422:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575902.071426:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575902.071431:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575902.071442:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575902/real 1788575902] req@ffff9ada60627480 x1875449616536704/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575927 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788575902.071444:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616536960/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575902.071489:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616536704/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575927 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575902.071501:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575902.071506:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575902.071509:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616536704:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575902.071538:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616536832:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575902.071561:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575902.071575:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575902/real 1788575902] req@ffff9ada60624e00 x1875449616536832/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575927 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575902.071588:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616536832/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575927 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575902.071595:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575902.071599:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575902.071602:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616536832:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575902.071608:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616536960:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575902.071632:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575902.071644:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575902/real 1788575902] req@ffff9ada60625c00 x1875449616536960/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575927 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575902.071669:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616536960/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575927 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575902.071677:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575902.071681:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575902.071684:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616536960:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575902.262995:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575902.583303:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575903.286995:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575903.607139:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575904.310984:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575904.631071:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575905.335691:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575905.655123:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575906.359032:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575906.679111:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575907.190986:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575907.190996:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575907.191001:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575907.191012:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7819 00000100:00080000:2.0:1788575907.191026:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575907.191034:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575907.191090:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616537088/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575907.191124:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575907.191129:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575907.191131:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575907.191135:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7819 00000100:00080000:2.0:1788575907.191139:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575907.191142:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575907.191152:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575907.191163:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616537216/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575907.191173:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575907.191177:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575907.191179:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575907.191182:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7819 00000100:00080000:2.0:1788575907.191186:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575907.191191:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575907.191197:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575907.191207:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616537344/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575907.191239:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616537088:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788575907.191327:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575907.191365:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575907/real 1788575907] req@ffff9ada60624380 x1875449616537088/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575932 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575907.191384:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616537088/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575932 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575907.191395:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575907.191400:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575907.191404:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616537088:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575907.191411:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616537216:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575907.191437:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575907.191450:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575907/real 1788575907] req@ffff9ada60625880 x1875449616537216/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575932 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575907.191463:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616537216/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575932 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575907.191472:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575907.191475:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575907.191478:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616537216:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575907.191485:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616537344:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575907.191506:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575907.191517:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575907/real 1788575907] req@ffff9ada60624a80 x1875449616537344/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575932 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575907.191529:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616537344/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575932 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575907.191537:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575907.191540:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575907.191543:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616537344:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575907.383281:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575907.703074:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575908.406974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575908.727003:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575909.431000:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575909.751693:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575910.456045:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575910.777206:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575911.479017:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575911.799102:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575912.311086:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575912.311099:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575912.311104:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575912.311123:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7824 00000100:00080000:1.0:1788575912.311137:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575912.311145:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575912.311194:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ed80 x1875449616537472/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575912.311229:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575912.311233:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575912.311235:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575912.311239:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7824 00000100:00080000:1.0:1788575912.311243:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575912.311247:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575912.311269:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575912.311284:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566dc00 x1875449616537600/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575912.311294:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575912.311298:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788575912.311299:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575912.311303:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7824 00000100:00080000:1.0:1788575912.311306:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575912.311311:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575912.311315:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575912.311325:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f800 x1875449616537728/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575912.311436:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616537472:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575912.311559:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616537600:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575912.311589:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616537728:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575912.503130:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575912.823101:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575913.526967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575913.847878:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575914.552782:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575914.871082:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1788575915.383164:0:2373:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:1.0:1788575915.383213:0:2373:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1788575915.383231:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:1.0:1788575915.383284:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:1.0:1788575915.383296:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:3.0:1788575915.383426:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575912/real 1788575915] req@ffff9ada4566ed80 x1875449616537472/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575937 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575915.383508:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ed80 x1875449616537472/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575937 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575915.383522:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575915.383528:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575915.383533:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616537472:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575915.383543:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575912/real 1788575915] req@ffff9ada4566dc00 x1875449616537600/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575937 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575915.383563:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566dc00 x1875449616537600/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575937 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575915.383574:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575915.383580:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575915.383584:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616537600:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575915.383592:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575912/real 1788575915] req@ffff9ada4566f800 x1875449616537728/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575937 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575915.383625:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f800 x1875449616537728/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575937 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575915.383635:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575915.383641:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575915.383646:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616537728:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575915.574971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575915.895133:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575916.598969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575916.919240:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575917.431011:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575917.431019:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575917.431023:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575917.431033:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7829 00000100:00080000:1.0:1788575917.431044:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575917.431050:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575917.431081:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e300 x1875449616537856/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575917.431114:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575917.431118:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575917.431120:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575917.431123:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7829 00000100:00080000:1.0:1788575917.431127:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575917.431131:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575917.431139:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575917.431151:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f480 x1875449616537984/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575917.431160:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575917.431164:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788575917.431165:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575917.431169:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7829 00000100:00080000:1.0:1788575917.431172:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1788575917.431175:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616537856:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788575917.431178:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575917.431186:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575917.431198:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ea00 x1875449616538112/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575917.431240:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575917.431272:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575917/real 1788575917] req@ffff9ada4566e300 x1875449616537856/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575942 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575917.431290:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e300 x1875449616537856/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575942 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575917.431300:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575917.431304:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575917.431308:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616537856:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575917.431315:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616537984:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575917.431337:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575917.431348:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575917/real 1788575917] req@ffff9ada4566f480 x1875449616537984/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575942 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575917.431361:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f480 x1875449616537984/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575942 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575917.431368:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575917.431371:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575917.431388:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616537984:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575917.431410:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616538112:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575917.431443:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575917.431462:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575917/real 1788575917] req@ffff9ada4566ea00 x1875449616538112/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575942 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575917.431475:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ea00 x1875449616538112/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575942 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575917.431482:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575917.431486:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575917.431488:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616538112:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575917.622960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575917.943163:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575918.646977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575918.967025:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575919.670988:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575919.991662:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575920.695001:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575921.015073:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575921.718970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575922.039057:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575922.550993:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575922.551012:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575922.551016:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575922.551027:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7835 00000100:00080000:0.0:1788575922.551037:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575922.551045:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575922.551090:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616538240/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575922.551123:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575922.551127:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575922.551129:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575922.551133:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7835 00000100:00080000:0.0:1788575922.551137:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575922.551141:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575922.551149:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575922.551161:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616538368/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575922.551170:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575922.551173:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575922.551175:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575922.551178:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7835 00000100:00080000:0.0:1788575922.551181:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575922.551184:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575922.551188:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575922.551197:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616538496/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575922.552942:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616538240:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788575922.553002:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575922.553043:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575922/real 1788575922] req@ffff9ada4557dc00 x1875449616538240/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575947 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575922.553062:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616538240/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575947 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575922.553072:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575922.553077:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575922.553081:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616538240:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575922.553088:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616538368:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575922.553111:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575922.553124:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575922/real 1788575922] req@ffff9ada4557d880 x1875449616538368/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575947 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575922.553136:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616538368/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575947 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575922.553150:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575922.553153:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575922.553156:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616538368:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575922.553163:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616538496:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575922.553182:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575922.553192:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575922/real 1788575922] req@ffff9ada4557ca80 x1875449616538496/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575947 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575922.553203:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616538496/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575947 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575922.553211:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575922.553213:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575922.553216:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616538496:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575922.742971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575923.063096:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575923.766962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575924.087044:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575924.790960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575925.111226:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575925.814956:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575926.134994:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575926.838963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575927.158993:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575927.670973:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575927.670980:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575927.670986:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575927.670995:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7840 00000100:00080000:0.0:1788575927.671003:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575927.671008:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575927.671038:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616538624/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575927.671061:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575927.671063:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575927.671064:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575927.671066:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7840 00000100:00080000:0.0:1788575927.671068:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575927.671070:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575927.671076:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575927.671083:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ce00 x1875449616538752/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575927.671091:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575927.671094:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575927.671096:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575927.671100:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7840 00000100:00080000:0.0:1788575927.671104:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575927.671108:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788575927.671112:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616538624:192.168.202.113@tcp:250:kworker.0 00000008:00100000:0.0:1788575927.671115:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575927.671131:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616538880/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575927.671178:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575927.671213:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575927/real 1788575927] req@ffff9ada4557c700 x1875449616538624/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575952 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575927.671227:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616538624/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575952 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575927.671235:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575927.671238:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575927.671240:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616538624:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575927.671245:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616538752:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575927.671261:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575927.671269:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575927/real 1788575927] req@ffff9ada4557ce00 x1875449616538752/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575952 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575927.671276:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ce00 x1875449616538752/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575952 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575927.671281:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575927.671283:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575927.671284:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616538752:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575927.671304:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616538880:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575927.671320:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575927.671330:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575927/real 1788575927] req@ffff9ada4557f480 x1875449616538880/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575952 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575927.671374:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616538880/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575952 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575927.671382:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575927.671385:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575927.671388:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616538880:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575927.862962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575928.182993:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575928.887002:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575929.206975:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575929.910958:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575930.231000:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575930.934937:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575931.255050:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575931.958940:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575932.279118:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575932.791024:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575932.791034:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575932.791038:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575932.791047:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7845 00000100:00080000:1.0:1788575932.791059:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575932.791066:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575932.791101:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d500 x1875449616539008/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575932.791125:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575932.791128:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575932.791134:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575932.791138:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7845 00000100:00080000:1.0:1788575932.791142:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575932.791146:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575932.791155:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575932.791169:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ca80 x1875449616539136/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575932.791177:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616539008:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788575932.791191:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575932.791196:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788575932.791198:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575932.791202:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7845 00000100:00080000:1.0:1788575932.791206:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575932.791210:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575932.791215:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575932.791226:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d180 x1875449616539264/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575932.791240:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575932.791277:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575932/real 1788575932] req@ffff9ada4566d500 x1875449616539008/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575957 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575932.791322:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d500 x1875449616539008/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575957 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575932.791338:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575932.791345:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575932.791349:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616539008:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575932.791379:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616539136:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575932.791403:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575932.791413:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575932/real 1788575932] req@ffff9ada4566ca80 x1875449616539136/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575957 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575932.791425:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ca80 x1875449616539136/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575957 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575932.791432:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575932.791436:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575932.791439:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616539136:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575932.791444:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616539264:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575932.791459:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575932.791468:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575932/real 1788575932] req@ffff9ada4566d180 x1875449616539264/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575957 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575932.791479:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d180 x1875449616539264/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575957 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575932.791487:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575932.791490:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575932.791493:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616539264:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575932.982940:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575933.303032:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575934.006954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575934.330480:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575935.030993:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575935.351082:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575936.055133:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575936.375016:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575937.078976:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575937.399158:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575937.911019:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575937.911032:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575937.911038:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575937.911047:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7850 00000100:00080000:0.0:1788575937.911059:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575937.911066:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575937.911107:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c000 x1875449616539392/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575937.911165:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575937.911170:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575937.911172:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575937.911176:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7850 00000100:00080000:0.0:1788575937.911179:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575937.911183:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575937.911195:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575937.911214:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c380 x1875449616539520/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575937.911227:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1788575937.911227:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616539392:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788575937.911248:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575937.911250:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575937.911256:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7850 00000100:00080000:0.0:1788575937.911259:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575937.911264:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575937.911269:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575937.911284:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616539648/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575937.911329:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575937.911375:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575937/real 1788575937] req@ffff9ada4557c000 x1875449616539392/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575962 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575937.911398:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c000 x1875449616539392/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575962 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575937.911409:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575937.911413:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575937.911417:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616539392:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575937.911453:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616539520:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575937.911480:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575937.911495:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575937/real 1788575937] req@ffff9ada4557c380 x1875449616539520/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575962 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575937.911508:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c380 x1875449616539520/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575962 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575937.911526:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575937.911530:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575937.911533:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616539520:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575937.911542:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616539648:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575937.911563:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575937.911574:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575937/real 1788575937] req@ffff9ada4557f480 x1875449616539648/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575962 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575937.911586:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616539648/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575962 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575937.911594:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575937.911597:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575937.911600:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616539648:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575938.103018:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575938.422987:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575939.126989:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575939.447057:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575940.150978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575940.471030:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575941.174964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575941.495014:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575942.198968:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575942.519033:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575943.031740:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575943.031750:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575943.031754:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575943.031772:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7855 00000100:00080000:1.0:1788575943.031782:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575943.031789:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575943.031825:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f100 x1875449616539776/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575943.031892:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575943.031896:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575943.031907:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1788575943.031912:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616539776:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788575943.031914:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7855 00000100:00080000:1.0:1788575943.031921:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575943.031927:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000800:00000100:3.0:1788575943.031979:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575943.032017:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575943/real 1788575943] req@ffff9ada4566f100 x1875449616539776/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575968 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575943.032035:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f100 x1875449616539776/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575968 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575943.032045:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575943.032049:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575943.032053:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616539776:192.168.202.113@tcp:250:kworker.0 00000008:00100000:1.0:1788575943.032077:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575943.032095:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c700 x1875449616539904/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575943.032113:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575943.032117:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788575943.032119:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575943.032123:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7855 00000100:00080000:1.0:1788575943.032128:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575943.032132:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575943.032138:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575943.032142:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616539904:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788575943.032152:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c380 x1875449616540032/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575943.032174:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575943.032203:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575943/real 1788575943] req@ffff9ada4566c700 x1875449616539904/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575968 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575943.032215:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c700 x1875449616539904/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575968 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575943.032223:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575943.032226:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575943.032230:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616539904:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575943.032241:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616540032:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575943.032261:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575943.032271:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575943/real 1788575943] req@ffff9ada4566c380 x1875449616540032/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575968 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575943.032283:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c380 x1875449616540032/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575968 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575943.032291:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575943.032308:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575943.032312:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616540032:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575943.222980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575943.543012:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575944.246970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575944.566991:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575945.270960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575945.591036:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575946.294964:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575946.615519:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575947.319290:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575947.639027:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575948.150990:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575948.150999:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575948.151004:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575948.151014:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7860 00000100:00080000:0.0:1788575948.151024:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575948.151039:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575948.151076:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ce00 x1875449616540160/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575948.151099:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575948.151102:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575948.151104:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575948.151107:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7860 00000100:00080000:0.0:1788575948.151110:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575948.151114:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575948.151123:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575948.151134:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616540288/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575948.151166:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616540160:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788575948.151173:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575948.151177:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575948.151179:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575948.151183:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7860 00000100:00080000:0.0:1788575948.151187:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575948.151190:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575948.151195:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575948.151207:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616540416/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575948.151228:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575948.151265:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575948/real 1788575948] req@ffff9ada4557ce00 x1875449616540160/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575973 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575948.151381:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ce00 x1875449616540160/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575973 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575948.151393:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575948.151396:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575948.151400:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616540160:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575948.151421:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616540288:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575948.151441:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575948.151452:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575948/real 1788575948] req@ffff9ada4557c700 x1875449616540288/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575973 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575948.151463:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616540288/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575973 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575948.151470:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575948.151473:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575948.151477:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616540288:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575948.151482:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616540416:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575948.151498:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575948.151506:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575948/real 1788575948] req@ffff9ada4557ca80 x1875449616540416/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575973 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575948.151516:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616540416/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575973 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575948.151523:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575948.151525:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575948.151528:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616540416:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575948.342951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575948.663047:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575949.366939:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575949.689272:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575950.390961:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788575950.711235:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575951.415177:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575951.736068:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575952.438992:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575952.759036:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575953.270981:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575953.270993:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575953.270998:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575953.271009:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7865 00000100:00080000:0.0:1788575953.271020:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575953.271027:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575953.271069:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616540544/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575953.271116:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575953.271124:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575953.271126:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575953.271131:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7865 00000100:00080000:0.0:1788575953.271135:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575953.271140:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575953.271154:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575953.271170:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616540672/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575953.271173:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616540544:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788575953.271189:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575953.271194:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575953.271195:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575953.271199:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7865 00000100:00080000:0.0:1788575953.271213:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575953.271218:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575953.271224:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575953.271235:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616540800/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575953.271256:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575953.271295:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575953/real 1788575953] req@ffff9ada4557d880 x1875449616540544/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575978 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575953.271312:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616540544/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575978 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575953.271320:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575953.271323:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575953.271326:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616540544:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575953.271349:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616540672:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575953.271365:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575953.271373:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575953/real 1788575953] req@ffff9ada4557dc00 x1875449616540672/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575978 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575953.271384:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616540672/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575978 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575953.271391:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575953.271394:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575953.271397:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616540672:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575953.271403:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616540800:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575953.271421:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575953.271432:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575953/real 1788575953] req@ffff9ada4557f800 x1875449616540800/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575978 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575953.271441:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616540800/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575978 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575953.271446:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575953.271448:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575953.271451:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616540800:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575953.463005:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575953.783274:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575954.486960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575954.807057:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575955.513572:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575955.831047:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575956.535008:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575956.855367:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575957.559022:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575957.879099:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575958.390991:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575958.391006:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575958.391012:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575958.391024:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7870 00000100:00080000:2.0:1788575958.391038:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575958.391066:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575958.391123:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616540928/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575958.391156:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575958.391161:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575958.391163:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575958.391168:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7870 00000100:00080000:2.0:1788575958.391172:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575958.391176:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575958.391187:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575958.391200:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616541056/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575958.391209:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575958.391212:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575958.391214:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575958.391217:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7870 00000100:00080000:2.0:1788575958.391220:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575958.391224:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575958.391228:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575958.391252:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616541184/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575958.391788:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616540928:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788575958.391884:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575958.391921:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575958/real 1788575958] req@ffff9ada60625500 x1875449616540928/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575983 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575958.391939:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616540928/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575983 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575958.391955:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575958.391959:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575958.391963:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616540928:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575958.391970:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616541056:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575958.391999:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575958.392011:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575958/real 1788575958] req@ffff9ada60626a00 x1875449616541056/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575983 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575958.392022:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616541056/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575983 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575958.392034:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575958.392036:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575958.392039:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616541056:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575958.392044:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616541184:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575958.392070:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575958.392079:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575958/real 1788575958] req@ffff9ada60626680 x1875449616541184/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575983 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575958.392089:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616541184/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575983 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575958.392096:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575958.392099:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575958.392101:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616541184:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575958.583086:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575958.903446:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575959.607084:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575959.927259:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575960.630978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575960.951167:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575961.656404:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575961.975206:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575962.679529:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575963.000883:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575963.510987:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575963.510998:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575963.511016:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575963.511027:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7876 00000100:00080000:2.0:1788575963.511038:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575963.511045:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575963.511096:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624700 x1875449616541312/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575963.511196:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616541312:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788575963.511254:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00080000:2.0:1788575963.511330:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575963.511336:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575963.511338:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575963.511343:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7876 00000100:00080000:2.0:1788575963.511348:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575963.511351:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575963.511361:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575963.511375:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627100 x1875449616541440/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575963.511502:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575963/real 1788575963] req@ffff9ada60624700 x1875449616541312/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575988 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575963.511520:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624700 x1875449616541312/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575988 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575963.511529:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575963.511533:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575963.511537:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616541312:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575963.511559:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616541440:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575963.511582:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575963.511594:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575963/real 1788575963] req@ffff9ada60627100 x1875449616541440/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575988 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575963.511607:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627100 x1875449616541440/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575988 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575963.511615:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575963.511618:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575963.511621:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616541440:192.168.202.113@tcp:38:kworker.0 00000100:00080000:2.0:1788575963.511731:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575963.511736:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575963.511738:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575963.511742:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7876 00000100:00080000:2.0:1788575963.511746:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575963.511750:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575963.511756:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575963.511770:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627100 x1875449616541568/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575963.511813:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616541568:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575963.511837:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575963.513070:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575963/real 1788575963] req@ffff9ada60627100 x1875449616541568/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575988 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575963.513100:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627100 x1875449616541568/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575988 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575963.513113:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575963.513116:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575963.513120:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616541568:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575963.703111:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575964.023393:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575964.727014:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575965.047073:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575965.751016:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575966.071007:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575966.775051:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575967.095985:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575967.798938:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575968.119047:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575968.630920:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575968.630929:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575968.630933:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575968.630940:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7881 00000100:00080000:2.0:1788575968.630949:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575968.630965:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575968.630999:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624700 x1875449616541696/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575968.631027:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575968.631031:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575968.631033:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575968.631037:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7881 00000100:00080000:2.0:1788575968.631041:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575968.631044:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575968.631054:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575968.631066:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616541824/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575968.631074:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575968.631076:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1788575968.631086:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616541696:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788575968.631090:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575968.631097:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7881 00000100:00080000:2.0:1788575968.631101:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575968.631105:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575968.631110:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575968.631121:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616541952/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575968.631154:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575968.631186:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575968/real 1788575968] req@ffff9ada60624700 x1875449616541696/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575993 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575968.631202:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624700 x1875449616541696/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575993 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575968.631212:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575968.631216:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575968.631220:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616541696:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575968.631226:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616541824:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575968.631251:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575968.631261:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575968/real 1788575968] req@ffff9ada60626680 x1875449616541824/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575993 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575968.631273:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616541824/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575993 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575968.631281:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575968.631283:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575968.631286:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616541824:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575968.631307:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616541952:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575968.631321:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575968.631330:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575968/real 1788575968] req@ffff9ada60626a00 x1875449616541952/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575993 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575968.631340:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616541952/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575993 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575968.631347:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575968.631350:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575968.631353:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616541952:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575968.823025:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575969.143315:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575969.847008:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575970.167173:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575970.871299:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575971.191166:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575971.895041:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575972.218125:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575972.919012:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575973.239030:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575973.750955:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575973.750966:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575973.750970:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575973.750980:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7886 00000100:00080000:2.0:1788575973.750992:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575973.751000:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575973.751035:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616542080/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575973.751069:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575973.751073:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575973.751074:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575973.751078:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7886 00000100:00080000:2.0:1788575973.751082:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575973.751085:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575973.751096:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575973.751109:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616542208/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575973.751119:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575973.751124:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575973.751127:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1788575973.751128:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616542080:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788575973.751133:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7886 00000100:00080000:2.0:1788575973.751138:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575973.751142:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575973.751152:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575973.751165:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616542336/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575973.751192:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575973.751225:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575973/real 1788575973] req@ffff9ada60625500 x1875449616542080/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575998 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575973.751244:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616542080/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788575998 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575973.751254:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575973.751258:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575973.751262:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616542080:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575973.751285:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616542208:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575973.751307:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575973.751320:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575973/real 1788575973] req@ffff9ada60624a80 x1875449616542208/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575998 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575973.751333:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616542208/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575998 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575973.751341:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575973.751344:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575973.751347:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616542208:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575973.751353:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616542336:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575973.751373:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575973.751384:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575973/real 1788575973] req@ffff9ada60625880 x1875449616542336/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575998 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575973.751402:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616542336/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788575998 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575973.751410:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575973.751413:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575973.751416:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616542336:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575973.942973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575974.263035:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575974.966998:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575975.287042:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575975.990971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575976.311062:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575977.014963:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575977.335017:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575978.040095:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575978.359013:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575978.870956:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575978.870967:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575978.870971:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575978.870982:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7891 00000100:00080000:0.0:1788575978.870993:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575978.871001:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575978.871065:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616542464/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575978.871094:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575978.871105:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575978.871107:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575978.871111:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7891 00000100:00080000:0.0:1788575978.871115:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575978.871119:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575978.871128:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575978.871143:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616542592/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575978.871150:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616542464:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788575978.871158:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575978.871163:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575978.871165:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575978.871169:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7891 00000100:00080000:0.0:1788575978.871174:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575978.871177:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575978.871183:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575978.871195:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616542720/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575978.871248:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616542592:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575978.871281:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616542720:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575979.062972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788575979.383079:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575980.086969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575980.407155:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575981.110991:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575981.431017:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1788575981.943134:0:2371:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:2.0:1788575981.943176:0:2371:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1788575981.943191:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788575981.943246:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788575981.943255:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:3.0:1788575981.943355:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575978/real 1788575981] req@ffff9ada4557fb80 x1875449616542592/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576003 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575981.943386:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616542592/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576003 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575981.943399:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575981.943404:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575981.943408:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616542592:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575981.943416:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575978/real 1788575981] req@ffff9ada4557ed80 x1875449616542720/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576003 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575981.943428:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616542720/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576003 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575981.943435:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575981.943439:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575981.943442:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616542720:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575981.943449:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575978/real 1788575981] req@ffff9ada4557e300 x1875449616542464/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576003 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575981.943462:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616542464/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576003 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575981.943470:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575981.943473:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575981.943476:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616542464:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575982.134962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575982.455096:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575983.158987:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575983.479075:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575983.990978:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575983.990985:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575983.990988:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575983.990995:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7896 00000100:00080000:0.0:1788575983.991004:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575983.991009:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575983.991034:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616542848/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575983.991055:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575983.991058:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575983.991060:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575983.991064:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7896 00000100:00080000:0.0:1788575983.991067:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575983.991070:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575983.991079:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575983.991090:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d180 x1875449616542976/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575983.991100:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575983.991104:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1788575983.991106:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616542848:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788575983.991107:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575983.991115:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7896 00000100:00080000:0.0:1788575983.991119:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575983.991122:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575983.991127:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575983.991138:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d500 x1875449616543104/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575983.991171:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575983.991227:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575983/real 1788575983] req@ffff9ada4557e680 x1875449616542848/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576008 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575983.991244:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616542848/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576008 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575983.991252:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575983.991260:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575983.991264:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616542848:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575983.991285:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616542976:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575983.991305:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575983.991315:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575983/real 1788575983] req@ffff9ada4557d180 x1875449616542976/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576008 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575983.991325:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d180 x1875449616542976/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576008 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575983.991333:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575983.991336:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575983.991339:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616542976:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575983.991343:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616543104:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575983.991361:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575983.991369:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575983/real 1788575983] req@ffff9ada4557d500 x1875449616543104/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576008 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575983.991378:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d500 x1875449616543104/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576008 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575983.991385:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575983.991388:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575983.991390:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616543104:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575984.182971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575984.503024:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575985.206957:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575985.527090:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575986.231120:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575986.551033:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575987.254948:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575987.575015:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575988.278970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575988.599038:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788575989.111042:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575989.111057:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788575989.111062:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575989.111075:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7901 00000100:00080000:2.0:1788575989.111089:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575989.111098:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788575989.111147:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616543232/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575989.111181:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575989.111185:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788575989.111187:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575989.111193:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7901 00000100:00080000:2.0:1788575989.111197:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575989.111201:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575989.111215:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788575989.111225:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616543232:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788575989.111230:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616543360/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788575989.111242:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788575989.111246:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788575989.111248:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788575989.111252:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7901 00000100:00080000:2.0:1788575989.111256:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788575989.111260:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788575989.111265:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788575989.111278:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616543488/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575989.111294:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575989.111331:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575989/real 1788575989] req@ffff9ada60624380 x1875449616543232/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576014 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575989.111369:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616543232/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576014 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575989.111378:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575989.111382:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575989.111385:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616543232:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575989.111407:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616543360:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575989.111423:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575989.111431:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575989/real 1788575989] req@ffff9ada60625c00 x1875449616543360/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576014 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575989.111439:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616543360/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576014 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575989.111444:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575989.111446:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575989.111448:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616543360:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575989.111451:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616543488:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575989.111461:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575989.111466:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575989/real 1788575989] req@ffff9ada60624e00 x1875449616543488/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576014 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575989.111472:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616543488/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576014 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575989.111477:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575989.111479:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575989.111480:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616543488:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575989.303021:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575989.623083:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575990.327005:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575990.647043:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575991.350989:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575991.671051:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575992.375039:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575992.695059:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575993.398959:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575993.719097:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788575994.231007:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575994.231016:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788575994.231021:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575994.231033:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7906 00000100:00080000:0.0:1788575994.231043:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575994.231050:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788575994.231097:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557df80 x1875449616543616/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788575994.231125:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575994.231129:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788575994.231131:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575994.231134:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7906 00000100:00080000:0.0:1788575994.231137:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575994.231141:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575994.231151:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575994.231163:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616543744/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575994.231171:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616543616:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788575994.231177:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788575994.231181:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788575994.231183:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788575994.231187:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7906 00000100:00080000:0.0:1788575994.231190:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788575994.231193:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788575994.231198:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788575994.231210:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d500 x1875449616543872/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575994.231231:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575994.231266:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575994/real 1788575994] req@ffff9ada4557df80 x1875449616543616/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576019 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575994.231285:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557df80 x1875449616543616/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576019 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575994.231295:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575994.231299:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575994.231303:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616543616:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575994.231324:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616543744:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575994.231345:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575994.231357:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575994/real 1788575994] req@ffff9ada4557ea00 x1875449616543744/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576019 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575994.231368:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616543744/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576019 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575994.231373:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575994.231376:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575994.231377:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616543744:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575994.231381:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616543872:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575994.231394:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575994.231400:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575994/real 1788575994] req@ffff9ada4557d500 x1875449616543872/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576019 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575994.231407:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d500 x1875449616543872/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576019 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575994.231411:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575994.231413:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575994.231415:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616543872:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575994.422960:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575994.742976:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575995.446939:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575995.767023:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575996.470962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575996.790980:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575997.494962:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575997.815036:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788575998.518979:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575998.839231:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788575999.350995:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575999.351006:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788575999.351010:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575999.351020:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7911 00000100:00080000:1.0:1788575999.351029:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575999.351037:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788575999.351073:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ce00 x1875449616544000/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788575999.351102:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575999.351106:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788575999.351108:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575999.351111:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7911 00000100:00080000:1.0:1788575999.351115:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575999.351118:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575999.351128:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575999.351142:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d880 x1875449616544128/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788575999.351152:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616544000:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788575999.351155:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788575999.351161:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788575999.351163:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788575999.351167:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7911 00000100:00080000:1.0:1788575999.351170:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788575999.351174:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788575999.351179:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788575999.351191:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e680 x1875449616544256/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788575999.351220:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575999.351254:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575999/real 1788575999] req@ffff9ada4566ce00 x1875449616544000/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576024 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575999.351280:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ce00 x1875449616544000/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576024 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575999.351289:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575999.351293:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575999.351297:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616544000:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788575999.351319:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616544128:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575999.351338:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575999.351346:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575999/real 1788575999] req@ffff9ada4566d880 x1875449616544128/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576024 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575999.351354:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d880 x1875449616544128/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576024 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575999.351359:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575999.351361:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575999.351363:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616544128:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575999.351367:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616544256:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788575999.351376:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788575999.351382:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788575999/real 1788575999] req@ffff9ada4566e680 x1875449616544256/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576024 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788575999.351388:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e680 x1875449616544256/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576024 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788575999.351396:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788575999.351398:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788575999.351400:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616544256:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788575999.542948:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788575999.863370:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576000.566975:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576000.887456:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576001.590973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576001.910989:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576002.614982:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576002.935046:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576003.640297:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576003.959018:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788576004.470984:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576004.470991:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788576004.470994:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576004.471004:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7917 00000100:00080000:2.0:1788576004.471013:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576004.471021:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788576004.471054:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616544384/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576004.471076:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576004.471079:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788576004.471080:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576004.471084:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7917 00000100:00080000:2.0:1788576004.471087:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576004.471090:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576004.471099:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576004.471112:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616544512/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576004.471123:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1788576004.471124:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616544384:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788576004.471130:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788576004.471133:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576004.471136:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7917 00000100:00080000:2.0:1788576004.471138:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576004.471141:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576004.471145:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576004.471154:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616544640/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788576004.471189:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576004.471223:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576004/real 1788576004] req@ffff9ada60627480 x1875449616544384/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576029 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576004.471239:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616544384/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576029 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576004.471246:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576004.471248:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576004.471251:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616544384:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576004.471275:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616544512:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576004.471290:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576004.471299:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576004/real 1788576004] req@ffff9ada60625f80 x1875449616544512/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576029 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576004.471306:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616544512/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576029 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576004.471311:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576004.471313:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576004.471315:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616544512:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576004.471318:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616544640:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576004.471327:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576004.471332:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576004/real 1788576004] req@ffff9ada60625180 x1875449616544640/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576029 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576004.471338:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616544640/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576029 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576004.471342:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576004.471344:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576004.471345:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616544640:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576004.665394:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576004.983076:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576005.686975:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576006.007050:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576006.710952:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576007.031136:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576007.734988:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576008.055058:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576008.758975:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576009.079041:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576009.591069:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576009.591076:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576009.591080:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576009.591090:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7922 00000100:00080000:0.0:1788576009.591099:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576009.591106:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576009.591153:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d180 x1875449616544768/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576009.591183:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576009.591187:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576009.591189:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576009.591194:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7922 00000100:00080000:0.0:1788576009.591197:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576009.591201:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576009.591210:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576009.591222:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616544896/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576009.591232:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576009.591235:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576009.591237:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576009.591251:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7922 00000100:00080000:0.0:1788576009.591255:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576009.591259:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576009.591264:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576009.591275:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616545024/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576009.591413:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616544768:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576009.591459:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576009.591483:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576009/real 1788576009] req@ffff9ada4557d180 x1875449616544768/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576034 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576009.591495:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d180 x1875449616544768/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576034 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576009.591501:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576009.591504:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576009.591506:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616544768:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576009.591510:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616544896:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576009.591522:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576009.591529:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576009/real 1788576009] req@ffff9ada4557e680 x1875449616544896/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576034 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576009.591535:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616544896/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576034 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576009.591540:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576009.591542:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576009.591543:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616544896:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576009.591546:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616545024:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576009.591554:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576009.591560:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576009/real 1788576009] req@ffff9ada4557e300 x1875449616545024/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576034 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576009.591566:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616545024/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576034 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576009.591570:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576009.591572:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576009.591574:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616545024:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576009.783054:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576010.103070:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576010.807015:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576011.128760:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576011.830969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576012.151096:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576012.855039:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576013.175036:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576013.879048:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576014.199017:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576014.710973:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576014.710982:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576014.710985:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576014.710993:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7927 00000100:00080000:0.0:1788576014.711001:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576014.711007:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576014.711039:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616545152/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576014.711062:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576014.711066:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576014.711067:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576014.711071:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7927 00000100:00080000:0.0:1788576014.711075:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576014.711078:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576014.711087:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576014.711100:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616545280/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576014.711109:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576014.711112:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576014.711125:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576014.711129:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7927 00000100:00100000:3.0:1788576014.711132:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616545152:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788576014.711134:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576014.711137:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576014.711141:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576014.711150:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616545408/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788576014.711189:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576014.711221:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576014/real 1788576014] req@ffff9ada4557ed80 x1875449616545152/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576039 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576014.711239:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616545152/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576039 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576014.711249:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576014.711253:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576014.711257:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616545152:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576014.711263:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616545280:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576014.711286:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576014.711298:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576014/real 1788576014] req@ffff9ada4557fb80 x1875449616545280/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576039 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576014.711309:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616545280/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576039 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576014.711316:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576014.711318:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576014.711321:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616545280:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576014.711356:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616545408:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576014.711378:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576014.711389:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576014/real 1788576014] req@ffff9ada4557f800 x1875449616545408/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576039 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576014.711400:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616545408/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576039 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576014.711408:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576014.711411:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576014.711413:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616545408:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576014.902981:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576015.223067:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576015.926970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576016.247223:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576016.950928:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576017.270995:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576017.974974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576018.295138:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576018.998982:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576019.319040:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788576019.831135:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576019.831145:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788576019.831150:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576019.831160:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7932 00000100:00080000:2.0:1788576019.831169:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576019.831175:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788576019.831212:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616545536/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576019.831240:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576019.831244:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788576019.831246:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576019.831249:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7932 00000100:00080000:2.0:1788576019.831253:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576019.831257:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576019.831265:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576019.831289:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616545664/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576019.831299:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576019.831301:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788576019.831303:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576019.831308:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7932 00000100:00100000:3.0:1788576019.831308:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616545536:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788576019.831319:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576019.831324:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576019.831328:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576019.831339:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626300 x1875449616545792/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788576019.831374:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576019.831406:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576019/real 1788576019] req@ffff9ada60626d80 x1875449616545536/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576044 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576019.831424:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616545536/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576044 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576019.831434:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576019.831438:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576019.831442:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616545536:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576019.831464:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616545664:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576019.831486:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576019.831498:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576019/real 1788576019] req@ffff9ada60627b80 x1875449616545664/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576044 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576019.831509:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616545664/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576044 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576019.831514:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576019.831516:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576019.831519:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616545664:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576019.831523:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616545792:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576019.831536:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576019.831556:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576019/real 1788576019] req@ffff9ada60626300 x1875449616545792/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576044 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576019.831568:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626300 x1875449616545792/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576044 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576019.831576:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576019.831579:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576019.831582:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616545792:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576020.023942:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576020.343039:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576021.046973:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576021.367067:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576022.071028:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576022.391233:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576023.094997:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576023.417477:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576024.118974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576024.439056:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788576024.951480:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576024.951490:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788576024.951494:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576024.951504:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7937 00000100:00080000:2.0:1788576024.951514:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576024.951522:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788576024.951559:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627800 x1875449616545920/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576024.951593:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576024.951596:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788576024.951598:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576024.951602:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7937 00000100:00080000:2.0:1788576024.951605:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576024.951608:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576024.951616:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576024.951629:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626300 x1875449616546048/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576024.951639:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576024.951642:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788576024.951643:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576024.951647:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7937 00000100:00080000:2.0:1788576024.951650:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576024.951653:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576024.951658:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576024.951667:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616546176/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576024.953162:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616545920:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576024.953219:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576024.953250:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576024/real 1788576024] req@ffff9ada60627800 x1875449616545920/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576049 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576024.953274:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627800 x1875449616545920/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576049 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576024.953283:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576024.953287:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576024.953290:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616545920:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576024.953298:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616546048:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576024.953320:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576024.953332:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576024/real 1788576024] req@ffff9ada60626300 x1875449616546048/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576049 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576024.953343:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626300 x1875449616546048/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576049 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576024.953351:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576024.953354:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576024.953358:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616546048:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576024.953364:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616546176:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576024.953383:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576024.953394:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576024/real 1788576024] req@ffff9ada60627b80 x1875449616546176/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576049 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576024.953405:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616546176/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576049 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576024.953419:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576024.953422:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576024.953437:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616546176:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576025.143015:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576025.463287:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576026.166980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576026.489944:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576027.190988:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576027.511516:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576028.215008:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576028.535310:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576029.238980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576029.558992:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576030.071024:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576030.071033:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576030.071037:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576030.071045:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7942 00000100:00080000:0.0:1788576030.071055:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576030.071061:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576030.071096:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616546304/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576030.071125:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576030.071128:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576030.071130:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576030.071133:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7942 00000100:00080000:0.0:1788576030.071137:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576030.071141:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576030.071149:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576030.071159:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616546432/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576030.071169:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576030.071172:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576030.071174:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576030.071177:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7942 00000100:00080000:0.0:1788576030.071180:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576030.071183:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576030.071187:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576030.071195:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616546560/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576030.071244:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616546304:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576030.071300:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576030.071333:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576030/real 1788576030] req@ffff9ada4557dc00 x1875449616546304/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576055 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576030.071499:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616546304/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576055 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576030.071513:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576030.071517:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576030.071521:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616546304:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576030.071528:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616546432:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576030.071552:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576030.071565:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576030/real 1788576030] req@ffff9ada4557d880 x1875449616546432/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576055 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576030.071711:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616546432/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576055 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576030.071724:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576030.071728:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576030.071732:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616546432:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576030.071738:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616546560:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576030.071760:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576030.071773:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576030/real 1788576030] req@ffff9ada4557ca80 x1875449616546560/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576055 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576030.071784:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616546560/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576055 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576030.071792:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576030.071795:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576030.071798:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616546560:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576030.262965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576030.583020:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576031.287008:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576031.608002:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576032.311040:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788576032.633942:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576033.335021:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576033.655051:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576034.358985:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576034.679144:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788576035.191024:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576035.191038:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788576035.191043:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576035.191055:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7947 00000100:00080000:2.0:1788576035.191069:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576035.191077:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788576035.191123:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616546688/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576035.191158:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576035.191161:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788576035.191163:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576035.191167:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7947 00000100:00080000:2.0:1788576035.191170:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576035.191174:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576035.191184:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576035.191195:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616546816/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576035.191203:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576035.191206:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788576035.191208:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576035.191211:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7947 00000100:00080000:2.0:1788576035.191214:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576035.191218:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576035.191222:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576035.191231:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616546944/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576035.191331:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616546688:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576035.191410:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576035.191453:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576035/real 1788576035] req@ffff9ada60626d80 x1875449616546688/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576060 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576035.191474:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616546688/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576060 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576035.191497:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576035.191501:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576035.191505:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616546688:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576035.191512:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616546816:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576035.191535:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576035.191548:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576035/real 1788576035] req@ffff9ada60625180 x1875449616546816/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576060 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576035.191560:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616546816/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576060 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576035.191568:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576035.191571:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576035.191574:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616546816:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576035.191580:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616546944:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576035.191600:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576035.191610:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576035/real 1788576035] req@ffff9ada60625f80 x1875449616546944/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576060 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576035.191623:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616546944/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576060 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576035.191631:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576035.191634:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576035.191637:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616546944:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576035.382982:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788576035.703806:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576036.409321:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576036.727066:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576037.431027:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576037.751066:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576038.455019:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788576038.775132:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576039.478947:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576039.799506:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576040.311015:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576040.311024:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576040.311029:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576040.311038:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7952 00000100:00080000:0.0:1788576040.311048:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576040.311063:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576040.311098:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616547072/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576040.311122:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576040.311124:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576040.311126:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576040.311128:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7952 00000100:00080000:0.0:1788576040.311131:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576040.311133:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576040.311141:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576040.311150:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ce00 x1875449616547200/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576040.311157:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576040.311159:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576040.311160:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576040.311162:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7952 00000100:00080000:0.0:1788576040.311164:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576040.311166:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576040.311169:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576040.311176:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616547328/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576040.311178:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616547072:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576040.311251:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576040.311284:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576040/real 1788576040] req@ffff9ada4557c700 x1875449616547072/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576065 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576040.311308:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616547072/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576065 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576040.311318:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576040.311322:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576040.311325:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616547072:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576040.311332:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616547200:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576040.311357:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576040.311368:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576040/real 1788576040] req@ffff9ada4557ce00 x1875449616547200/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576065 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576040.311378:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ce00 x1875449616547200/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576065 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576040.311385:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576040.311388:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576040.311390:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616547200:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576040.311410:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616547328:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576040.311426:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576040.311435:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576040/real 1788576040] req@ffff9ada4557f480 x1875449616547328/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576065 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576040.311444:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616547328/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576065 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576040.311451:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576040.311453:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576040.311455:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616547328:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576040.503025:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576040.823096:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576041.527007:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788576041.848016:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576042.550981:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576042.871064:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576043.575007:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576043.895225:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576044.598974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788576044.919119:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788576045.431015:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576045.431026:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788576045.431030:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576045.431041:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7957 00000100:00080000:2.0:1788576045.431051:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576045.431058:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788576045.431092:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616547456/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576045.431125:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576045.431129:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788576045.431130:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576045.431134:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7957 00000100:00080000:2.0:1788576045.431138:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576045.431142:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576045.431150:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576045.431162:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616547584/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576045.431172:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576045.431175:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788576045.431177:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576045.431180:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7957 00000100:00080000:2.0:1788576045.431183:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576045.431186:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576045.431191:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576045.431200:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616547712/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576045.431303:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616547456:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576045.431372:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616547584:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576045.431397:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616547712:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576045.622987:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576045.943923:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576046.648219:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576046.967077:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576047.670982:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788576047.991055:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1788576048.503179:0:2370:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:2.0:1788576048.503224:0:2370:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1788576048.503240:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788576048.503276:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788576048.503285:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:3.0:1788576048.503372:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576045/real 1788576048] req@ffff9ada60627480 x1875449616547456/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576070 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576048.503407:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616547456/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576070 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576048.503420:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576048.503426:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576048.503430:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616547456:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576048.503438:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576045/real 1788576048] req@ffff9ada60624e00 x1875449616547584/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576070 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576048.503450:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616547584/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576070 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576048.503458:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576048.503461:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576048.503465:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616547584:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576048.503472:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576045/real 1788576048] req@ffff9ada60625c00 x1875449616547712/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576070 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576048.503482:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616547712/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576070 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576048.503488:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576048.503490:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576048.503493:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616547712:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576048.695007:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576049.015080:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576049.718989:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576050.039128:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576050.551043:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576050.551056:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576050.551063:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576050.551075:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7963 00000100:00080000:0.0:1788576050.551088:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576050.551097:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576050.551145:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c380 x1875449616547840/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576050.551179:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576050.551183:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576050.551185:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576050.551190:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7963 00000100:00080000:0.0:1788576050.551193:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576050.551198:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576050.551210:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576050.551223:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c000 x1875449616547968/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576050.551235:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576050.551238:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576050.551241:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1788576050.551246:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616547840:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788576050.551247:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7963 00000100:00080000:0.0:1788576050.551253:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576050.551258:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576050.551274:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576050.551288:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616548096/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788576050.551340:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576050.551385:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576050/real 1788576050] req@ffff9ada4557c380 x1875449616547840/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576075 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576050.551408:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c380 x1875449616547840/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576075 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576050.551421:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576050.551425:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576050.551429:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616547840:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576050.551458:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616547968:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576050.551485:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576050.551497:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576050/real 1788576050] req@ffff9ada4557c000 x1875449616547968/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576075 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576050.551509:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c000 x1875449616547968/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576075 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576050.551517:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576050.551521:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576050.551524:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616547968:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576050.551530:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616548096:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576050.551550:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576050.551561:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576050/real 1788576050] req@ffff9ada4557f480 x1875449616548096/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576075 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576050.551572:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616548096/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576075 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576050.551580:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576050.551584:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576050.551587:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616548096:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576050.743242:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788576051.063036:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576051.767219:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576052.087040:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576052.790967:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576053.111043:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576053.814975:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576054.135046:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576054.840043:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576055.159133:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788576055.671017:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576055.671031:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788576055.671036:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576055.671048:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7968 00000100:00080000:1.0:1788576055.671062:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576055.671070:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788576055.671121:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e680 x1875449616548224/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788576055.671152:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576055.671156:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788576055.671158:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576055.671162:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7968 00000100:00080000:1.0:1788576055.671166:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576055.671171:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576055.671182:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788576055.671197:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616548224:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788576055.671198:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d880 x1875449616548352/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788576055.671214:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576055.671217:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788576055.671219:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576055.671223:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7968 00000100:00080000:1.0:1788576055.671227:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576055.671231:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576055.671235:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576055.671246:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ce00 x1875449616548480/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788576055.671278:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576055.671313:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576055/real 1788576055] req@ffff9ada4566e680 x1875449616548224/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576080 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576055.671329:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e680 x1875449616548224/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576080 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576055.671337:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576055.671340:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576055.671343:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616548224:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576055.671363:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616548352:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576055.671377:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576055.671385:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576055/real 1788576055] req@ffff9ada4566d880 x1875449616548352/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576080 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576055.671401:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d880 x1875449616548352/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576080 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576055.671409:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576055.671412:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576055.671415:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616548352:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576055.671421:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616548480:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576055.671440:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576055.671451:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576055/real 1788576055] req@ffff9ada4566ce00 x1875449616548480/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576080 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576055.671463:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ce00 x1875449616548480/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576080 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576055.671471:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576055.671483:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576055.671486:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616548480:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576055.863013:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576056.183096:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576056.887089:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576057.207097:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576057.910993:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576058.231044:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576058.935039:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576059.255064:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576059.958978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576060.279090:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788576060.790988:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576060.790997:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788576060.791003:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576060.791015:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7973 00000100:00080000:1.0:1788576060.791028:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576060.791038:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788576060.791086:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c380 x1875449616548608/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788576060.791119:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576060.791123:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788576060.791126:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576060.791129:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7973 00000100:00080000:1.0:1788576060.791132:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576060.791135:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576060.791146:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576060.791156:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c700 x1875449616548736/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788576060.791165:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576060.791168:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788576060.791170:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1788576060.791175:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616548608:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788576060.791176:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7973 00000100:00080000:1.0:1788576060.791183:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576060.791188:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576060.791193:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576060.791206:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f100 x1875449616548864/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788576060.791266:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576060.791305:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576060/real 1788576060] req@ffff9ada4566c380 x1875449616548608/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576085 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576060.791319:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c380 x1875449616548608/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576085 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576060.791327:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576060.791330:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576060.791333:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616548608:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576060.791358:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616548736:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576060.791373:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576060.791381:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576060/real 1788576060] req@ffff9ada4566c700 x1875449616548736/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576085 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576060.791388:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c700 x1875449616548736/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576085 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576060.791394:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576060.791396:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576060.791398:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616548736:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576060.791409:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616548864:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576060.791419:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576060.791425:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576060/real 1788576060] req@ffff9ada4566f100 x1875449616548864/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576085 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576060.791431:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f100 x1875449616548864/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576085 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576060.791436:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576060.791438:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576060.791439:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616548864:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576060.983037:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576061.303062:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576062.007005:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576062.328203:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576063.030993:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576063.351125:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576064.055001:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576064.375047:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576065.078988:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576065.399143:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788576065.911792:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576065.911802:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788576065.911807:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576065.911817:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7978 00000100:00080000:2.0:1788576065.911827:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576065.911835:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788576065.911894:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616548992/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576065.911952:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576065.911956:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788576065.911959:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576065.911963:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7978 00000100:00080000:2.0:1788576065.911967:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576065.911971:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788576065.911976:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616548992:192.168.202.113@tcp:250:kworker.0 00000008:00100000:2.0:1788576065.911982:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576065.911997:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616549120/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576065.912018:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576065.912021:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788576065.912023:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576065.912027:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7978 00000100:00080000:2.0:1788576065.912030:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576065.912034:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576065.912038:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576065.912049:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616549248/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788576065.912141:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576065.912170:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576065/real 1788576065] req@ffff9ada60624380 x1875449616548992/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576090 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576065.912187:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616548992/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576090 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576065.912198:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576065.912202:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576065.912205:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616548992:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576065.912225:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616549120:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576065.912243:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576065.912253:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576065/real 1788576065] req@ffff9ada60625880 x1875449616549120/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576090 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576065.912264:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616549120/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576090 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576065.912272:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576065.912275:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576065.912278:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616549120:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576065.912284:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616549248:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576065.912299:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576065.912308:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576065/real 1788576065] req@ffff9ada60624a80 x1875449616549248/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576090 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576065.912318:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616549248/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576090 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576065.912326:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576065.912379:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576065.912384:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616549248:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576066.104110:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576066.422989:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576067.126999:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576067.451547:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576068.150988:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576068.471055:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576069.174989:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576069.495052:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576070.198981:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576070.519966:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788576071.031054:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576071.031065:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788576071.031073:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576071.031083:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7983 00000100:00080000:1.0:1788576071.031095:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576071.031102:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788576071.031137:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d180 x1875449616549376/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788576071.031173:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576071.031177:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788576071.031178:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576071.031182:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7983 00000100:00080000:1.0:1788576071.031185:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576071.031188:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576071.031196:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576071.031207:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ca80 x1875449616549504/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788576071.031215:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576071.031218:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788576071.031220:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576071.031223:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7983 00000100:00080000:1.0:1788576071.031226:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576071.031229:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576071.031233:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576071.031253:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d500 x1875449616549632/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576071.032461:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616549376:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576071.032525:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576071.032558:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576071/real 1788576071] req@ffff9ada4566d180 x1875449616549376/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576096 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576071.032576:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d180 x1875449616549376/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576096 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576071.032586:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576071.032590:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576071.032593:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616549376:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576071.032601:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616549504:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576071.032622:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576071.032636:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576071/real 1788576071] req@ffff9ada4566ca80 x1875449616549504/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576096 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576071.032648:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ca80 x1875449616549504/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576096 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576071.032656:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576071.032659:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576071.032662:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616549504:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576071.032668:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616549632:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576071.032684:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576071.032694:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576071/real 1788576071] req@ffff9ada4566d500 x1875449616549632/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576096 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576071.032704:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d500 x1875449616549632/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576096 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576071.032712:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576071.032715:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576071.032718:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616549632:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576071.223122:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576071.543218:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576072.247001:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576072.568004:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576073.271068:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576073.591094:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576074.295032:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576074.615150:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576075.318990:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576075.639140:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788576076.150984:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576076.150993:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788576076.150997:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576076.151006:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7988 00000100:00080000:2.0:1788576076.151015:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576076.151022:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788576076.151055:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616549760/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576076.151080:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576076.151084:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788576076.151086:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576076.151089:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7988 00000100:00080000:2.0:1788576076.151092:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576076.151096:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576076.151104:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576076.151116:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616549888/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576076.151127:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576076.151131:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00100000:3.0:1788576076.151132:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616549760:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788576076.151134:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576076.151141:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7988 00000100:00080000:2.0:1788576076.151144:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576076.151148:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576076.151153:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576076.151162:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616550016/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788576076.151194:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576076.151224:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576076/real 1788576076] req@ffff9ada60625500 x1875449616549760/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576101 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576076.151242:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616549760/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576101 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576076.151252:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576076.151256:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576076.151259:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616549760:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576076.151280:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616549888:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576076.151299:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576076.151309:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576076/real 1788576076] req@ffff9ada60626a00 x1875449616549888/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576101 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576076.151320:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616549888/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576101 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576076.151333:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576076.151336:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576076.151344:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616549888:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576076.151350:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616550016:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576076.151367:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576076.151376:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576076/real 1788576076] req@ffff9ada60626680 x1875449616550016/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576101 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576076.151387:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616550016/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576101 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576076.151394:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576076.151397:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576076.151400:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616550016:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576076.342972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576076.663031:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576077.367001:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576077.687240:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576078.390947:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576078.711138:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576079.416920:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576079.736416:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576080.439017:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576080.761303:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576081.271039:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576081.271050:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576081.271054:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576081.271064:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7993 00000100:00080000:0.0:1788576081.271074:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576081.271082:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576081.271129:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ce00 x1875449616550144/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576081.271152:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576081.271155:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576081.271156:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576081.271159:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7993 00000100:00080000:0.0:1788576081.271161:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576081.271164:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576081.271171:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576081.271178:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616550272/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576081.271184:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576081.271186:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576081.271187:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576081.271189:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7993 00000100:00080000:0.0:1788576081.271191:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576081.271192:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576081.271195:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576081.271201:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616550400/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576081.272172:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616550144:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576081.272234:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576081.272267:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576081/real 1788576081] req@ffff9ada4557ce00 x1875449616550144/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576106 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576081.272284:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ce00 x1875449616550144/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576106 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576081.272293:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576081.272297:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576081.272300:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616550144:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576081.272306:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616550272:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576081.272337:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576081.272349:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576081/real 1788576081] req@ffff9ada4557c700 x1875449616550272/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576106 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576081.272361:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616550272/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576106 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576081.272369:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576081.272371:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576081.272374:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616550272:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576081.272380:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616550400:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576081.272398:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576081.272408:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576081/real 1788576081] req@ffff9ada4557ca80 x1875449616550400/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576106 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576081.272424:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616550400/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576106 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576081.272432:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576081.272435:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576081.272438:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616550400:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576081.464125:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576081.783653:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576082.487043:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576082.807074:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576083.511733:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576083.834004:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576084.535034:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576084.858081:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576085.559018:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576085.881288:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788576086.391035:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576086.391106:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788576086.391167:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576086.391182:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 7998 00000100:00080000:2.0:1788576086.391195:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576086.391204:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788576086.391254:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624700 x1875449616550528/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576086.391288:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576086.391293:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788576086.391295:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576086.391299:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7998 00000100:00080000:2.0:1788576086.391303:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576086.391308:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576086.391319:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576086.391331:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627100 x1875449616550656/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576086.391341:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576086.391344:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788576086.391345:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576086.391349:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 7998 00000100:00080000:2.0:1788576086.391352:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576086.391356:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576086.391360:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576086.391374:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616550784/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576086.391466:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616550528:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576086.391543:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576086.391599:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576086/real 1788576086] req@ffff9ada60624700 x1875449616550528/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576111 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576086.391621:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624700 x1875449616550528/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576111 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576086.391633:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576086.391638:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576086.391642:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616550528:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576086.391661:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616550656:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576086.391687:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576086.391700:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576086/real 1788576086] req@ffff9ada60627100 x1875449616550656/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576111 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576086.391712:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627100 x1875449616550656/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576111 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576086.391719:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576086.391723:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576086.391726:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616550656:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576086.391732:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616550784:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576086.391752:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576086.391764:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576086/real 1788576086] req@ffff9ada60626680 x1875449616550784/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576111 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576086.392015:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616550784/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576111 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576086.392025:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576086.392028:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576086.392033:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616550784:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576086.583005:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576086.903125:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576087.606958:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576087.928317:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576088.631003:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576088.951100:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576089.654955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576089.975070:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576090.678955:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576090.999062:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788576091.511001:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576091.511011:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788576091.511016:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576091.511027:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8004 00000100:00080000:1.0:1788576091.511039:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576091.511048:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788576091.511085:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ea00 x1875449616550912/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788576091.511113:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576091.511116:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788576091.511117:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576091.511121:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8004 00000100:00080000:1.0:1788576091.511124:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576091.511128:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576091.511139:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788576091.511168:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616550912:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788576091.511226:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f480 x1875449616551040/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788576091.511235:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576091.511239:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000800:00000100:3.0:1788576091.511241:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00080000:1.0:1788576091.511242:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576091.511248:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8004 00000100:00080000:1.0:1788576091.511253:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576091.511258:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576091.511264:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788576091.511278:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576091/real 1788576091] req@ffff9ada4566ea00 x1875449616550912/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576116 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788576091.511279:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e300 x1875449616551168/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576091.511319:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ea00 x1875449616550912/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576116 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576091.511336:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576091.511348:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576091.511355:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616550912:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576091.511377:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616551040:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576091.511396:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576091.511406:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576091/real 1788576091] req@ffff9ada4566f480 x1875449616551040/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576116 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576091.511416:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f480 x1875449616551040/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576116 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576091.511423:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576091.511426:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576091.511428:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616551040:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576091.511434:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616551168:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576091.511446:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576091.511453:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576091/real 1788576091] req@ffff9ada4566e300 x1875449616551168/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576116 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576091.511462:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e300 x1875449616551168/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576116 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576091.511468:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576091.511471:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576091.511473:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616551168:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576091.703705:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576092.023077:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576092.727165:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576093.048724:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576093.750965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576094.071418:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576094.774987:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576095.096096:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576095.800410:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576096.119713:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788576096.631070:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576096.631080:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788576096.631084:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576096.631094:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8009 00000100:00080000:2.0:1788576096.631104:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576096.631112:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788576096.631157:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616551296/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576096.631251:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616551296:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576096.631310:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00080000:2.0:1788576096.631343:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576096.631347:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788576096.631349:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576096.631353:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8009 00000100:00080000:2.0:1788576096.631357:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576096.631360:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576096.631370:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576096.631384:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616551424/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576096.631483:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576096/real 1788576096] req@ffff9ada60626a00 x1875449616551296/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576121 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576096.631501:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616551296/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576121 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576096.631511:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576096.631514:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576096.631517:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616551296:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576096.631539:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616551424:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576096.631560:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576096.631571:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576096/real 1788576096] req@ffff9ada60625500 x1875449616551424/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576121 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576096.631583:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616551424/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576121 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576096.631590:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576096.631593:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576096.631595:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616551424:192.168.202.113@tcp:38:kworker.0 00000100:00080000:2.0:1788576096.631689:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576096.631693:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788576096.631695:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576096.631699:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8009 00000100:00080000:2.0:1788576096.631704:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576096.631707:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576096.631713:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576096.631726:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616551552/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576096.631770:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616551552:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576096.631803:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576096.631977:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576096/real 1788576096] req@ffff9ada60624a80 x1875449616551552/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576121 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576096.631989:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616551552/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576121 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576096.632006:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576096.632010:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576096.632013:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616551552:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576096.822953:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576097.145277:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576097.849281:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576098.167129:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576098.871459:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576099.191122:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576099.895163:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576100.215068:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576100.918958:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576101.242904:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788576101.751317:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576101.751326:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788576101.751331:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576101.751341:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8014 00000100:00080000:2.0:1788576101.751351:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576101.751359:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788576101.751396:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616551680/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576101.751430:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576101.751434:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788576101.751436:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576101.751439:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8014 00000100:00080000:2.0:1788576101.751443:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576101.751446:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576101.751456:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576101.751468:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616551808/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576101.751477:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576101.751480:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788576101.751481:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576101.751485:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8014 00000100:00080000:2.0:1788576101.751488:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576101.751492:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576101.751496:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576101.751506:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616551936/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576101.751681:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616551680:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576101.751741:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576101.751790:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576101/real 1788576101] req@ffff9ada60625880 x1875449616551680/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576126 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576101.751808:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616551680/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576126 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576101.751818:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576101.751822:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576101.751826:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616551680:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576101.751833:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616551808:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576101.751875:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576101.751887:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576101/real 1788576101] req@ffff9ada60624380 x1875449616551808/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576126 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576101.751898:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616551808/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576126 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576101.751911:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576101.751914:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576101.751917:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616551808:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576101.751923:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616551936:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576101.751941:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576101.751951:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576101/real 1788576101] req@ffff9ada60625c00 x1875449616551936/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576126 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576101.751962:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616551936/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576126 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576101.751969:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576101.751972:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576101.751975:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616551936:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576101.943199:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576102.262999:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576102.966924:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576103.287539:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576103.991353:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576104.311919:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576105.014966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576105.335226:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576106.038978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576106.359336:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576106.871456:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576106.871470:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576106.871476:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576106.871490:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8019 00000100:00080000:0.0:1788576106.871501:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576106.871509:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576106.871556:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616552064/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576106.871586:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576106.871590:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576106.871592:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576106.871596:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8019 00000100:00080000:0.0:1788576106.871599:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576106.871603:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576106.871614:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576106.871624:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616552192/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576106.871631:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576106.871634:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576106.871635:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576106.871638:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8019 00000100:00080000:0.0:1788576106.871640:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576106.871642:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576106.871645:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576106.871650:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616552320/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576106.871958:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616552064:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576106.872042:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576106.873260:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576106/real 1788576106] req@ffff9ada4557d880 x1875449616552064/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576131 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576106.873293:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616552064/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576131 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576106.873302:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576106.873305:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576106.873308:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616552064:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576106.873315:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616552192:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576106.873347:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576106.873359:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576106/real 1788576106] req@ffff9ada4557dc00 x1875449616552192/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576131 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576106.873368:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616552192/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576131 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576106.873372:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576106.873374:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576106.873376:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616552192:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576106.873380:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616552320:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576106.873391:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576106.873396:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576106/real 1788576106] req@ffff9ada4557f800 x1875449616552320/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576131 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576106.873403:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616552320/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576131 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576106.873412:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576106.873414:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576106.873416:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616552320:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576107.063215:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576107.384051:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576108.087570:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576108.408442:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576109.110983:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576109.431024:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576110.137459:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576110.455197:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576111.158980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576111.480188:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788576111.992896:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576111.992906:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788576111.992911:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576111.992921:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8024 00000100:00080000:1.0:1788576111.992943:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576111.992951:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788576111.992985:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f800 x1875449616552448/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788576111.993013:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576111.993016:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788576111.993018:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576111.993021:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8024 00000100:00080000:1.0:1788576111.993024:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576111.993027:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576111.993045:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576111.993056:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566dc00 x1875449616552576/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788576111.993065:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576111.993068:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788576111.993070:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576111.993073:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8024 00000100:00080000:1.0:1788576111.993076:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576111.993079:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576111.993084:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576111.993091:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ed80 x1875449616552704/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576111.993133:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616552448:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576111.993213:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616552576:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576111.993236:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616552704:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576112.182981:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576112.503097:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576113.207298:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576113.528054:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576114.231795:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576114.551004:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1788576115.065544:0:2372:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:1.0:1788576115.065596:0:2372:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1788576115.065615:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:1.0:1788576115.065663:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:1.0:1788576115.065672:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:3.0:1788576115.065797:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576111/real 1788576115] req@ffff9ada4566f800 x1875449616552448/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576136 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576115.065821:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f800 x1875449616552448/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576136 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576115.065834:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576115.065839:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576115.065844:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616552448:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576115.065882:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576111/real 1788576115] req@ffff9ada4566dc00 x1875449616552576/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576136 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576115.065897:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566dc00 x1875449616552576/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576136 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576115.065906:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576115.065909:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576115.065912:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616552576:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576115.065918:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576111/real 1788576115] req@ffff9ada4566ed80 x1875449616552704/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576136 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576115.065930:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ed80 x1875449616552704/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576136 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576115.065937:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576115.065940:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576115.065943:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616552704:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576115.254980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576115.575211:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576116.278978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576116.600272:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788576117.111885:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576117.111897:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788576117.111901:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576117.111914:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8029 00000100:00080000:1.0:1788576117.111926:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576117.111933:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788576117.111968:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c000 x1875449616552832/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788576117.111998:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576117.112002:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788576117.112003:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576117.112007:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8029 00000100:00080000:1.0:1788576117.112010:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576117.112014:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576117.112026:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576117.112037:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566df80 x1875449616552960/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788576117.112045:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576117.112048:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788576117.112061:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576117.112066:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8029 00000100:00080000:1.0:1788576117.112069:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576117.112072:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576117.112077:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576117.112088:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566fb80 x1875449616553088/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576117.112113:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616552832:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576117.112176:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576117.112208:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576117/real 1788576117] req@ffff9ada4566c000 x1875449616552832/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576142 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576117.112226:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c000 x1875449616552832/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576142 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576117.112237:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576117.112241:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576117.112245:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616552832:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576117.112252:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616552960:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576117.112272:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576117.112284:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576117/real 1788576117] req@ffff9ada4566df80 x1875449616552960/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576142 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576117.112296:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566df80 x1875449616552960/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576142 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576117.112305:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576117.112308:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576117.112310:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616552960:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576117.112316:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616553088:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576117.112341:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576117.112352:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576117/real 1788576117] req@ffff9ada4566fb80 x1875449616553088/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576142 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576117.112362:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566fb80 x1875449616553088/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576142 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576117.112369:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576117.112371:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576117.112374:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616553088:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576117.302978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576117.623230:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576118.327112:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576118.647099:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576119.351039:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576119.671121:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576120.375201:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576120.695141:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576121.398949:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576121.719419:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576122.231326:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576122.231340:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576122.231345:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576122.231358:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8034 00000100:00080000:0.0:1788576122.231372:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576122.231381:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576122.231432:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616553216/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576122.231470:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576122.231474:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576122.231476:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576122.231480:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8034 00000100:00080000:0.0:1788576122.231484:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1788576122.231557:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616553216:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788576122.231604:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576122.231620:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576122.231638:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616553344/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788576122.231666:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576122.231709:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576122/real 1788576122] req@ffff9ada4557fb80 x1875449616553216/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576147 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788576122.231715:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576122.231727:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576122.231732:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576122.231743:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8034 00000100:00080000:0.0:1788576122.231750:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1788576122.231757:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616553216/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576147 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788576122.231758:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576122.231767:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00080000:3.0:1788576122.231778:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576122.231786:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788576122.231790:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616553472/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576122.231808:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616553216:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576122.231842:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616553344:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576122.232192:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576122.232204:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576122/real 1788576122] req@ffff9ada4557ed80 x1875449616553344/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576147 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576122.232229:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616553344/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576147 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576122.232237:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576122.232241:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576122.232244:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616553344:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576122.232251:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616553472:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576122.232271:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576122.232282:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576122/real 1788576122] req@ffff9ada4557e300 x1875449616553472/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576147 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576122.232300:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616553472/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576147 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576122.232308:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576122.232311:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576122.232314:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616553472:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576122.422965:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576122.743014:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576123.447171:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576123.767016:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576124.470980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576124.791059:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576125.496008:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576125.818686:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576126.518992:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576126.842885:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788576127.351131:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576127.351142:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788576127.351147:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576127.351157:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8039 00000100:00080000:1.0:1788576127.351168:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576127.351177:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788576127.351227:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566fb80 x1875449616553600/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788576127.351257:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576127.351262:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788576127.351264:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576127.351269:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8039 00000100:00080000:1.0:1788576127.351273:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576127.351278:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576127.351289:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576127.351302:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616553600:192.168.202.113@tcp:250:kworker.0 00000100:00100000:1.0:1788576127.351304:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566df80 x1875449616553728/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788576127.351318:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576127.351322:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788576127.351324:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576127.351329:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8039 00000100:00080000:1.0:1788576127.351332:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576127.351336:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576127.351341:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576127.351353:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c000 x1875449616553856/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1788576127.351374:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576127.351410:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576127/real 1788576127] req@ffff9ada4566fb80 x1875449616553600/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576152 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576127.351428:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566fb80 x1875449616553600/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576152 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576127.351440:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576127.351443:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576127.351446:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616553600:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788576127.351466:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616553728:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576127.351483:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576127.351491:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576127/real 1788576127] req@ffff9ada4566df80 x1875449616553728/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576152 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576127.351498:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566df80 x1875449616553728/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576152 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576127.351503:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576127.351505:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576127.351507:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616553728:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576127.351511:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616553856:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576127.351530:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576127.351538:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576127/real 1788576127] req@ffff9ada4566c000 x1875449616553856/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576152 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576127.351544:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c000 x1875449616553856/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576152 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576127.351549:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576127.351551:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576127.351553:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616553856:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576127.547429:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576127.863107:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576128.566944:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576128.887061:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576129.591386:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576129.911359:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576130.614999:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576130.935372:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576131.639054:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576131.959409:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788576132.470973:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576132.470984:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788576132.470989:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576132.470999:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8045 00000100:00080000:1.0:1788576132.471009:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576132.471017:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788576132.471052:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ed80 x1875449616553984/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788576132.471161:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616553984:192.168.202.113@tcp:250:kworker.0 00000800:00000100:2.0:1788576132.471229:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00080000:1.0:1788576132.471305:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576132.471309:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788576132.471311:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576132.471315:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8045 00000100:00080000:1.0:1788576132.471319:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576132.471323:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576132.471332:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576132.471346:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566dc00 x1875449616554112/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788576132.471448:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576132/real 1788576132] req@ffff9ada4566ed80 x1875449616553984/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576157 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576132.471463:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ed80 x1875449616553984/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576157 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576132.471472:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576132.471476:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576132.471479:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616553984:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788576132.471499:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616554112:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576132.471532:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576132.471544:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576132/real 1788576132] req@ffff9ada4566dc00 x1875449616554112/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576157 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576132.471554:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566dc00 x1875449616554112/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576157 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576132.471560:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576132.471563:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576132.471566:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616554112:192.168.202.113@tcp:38:kworker.0 00000100:00080000:1.0:1788576132.471822:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576132.471825:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788576132.471827:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576132.471831:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8045 00000100:00080000:1.0:1788576132.471835:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576132.471838:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576132.471843:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576132.471873:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f800 x1875449616554240/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788576132.471929:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616554240:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576132.471950:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576132.471961:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576132/real 1788576132] req@ffff9ada4566f800 x1875449616554240/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576157 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576132.471971:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f800 x1875449616554240/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576157 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576132.471977:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576132.471980:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576132.471982:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616554240:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576132.662977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788576132.983442:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576133.686985:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576134.007497:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576134.711178:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576135.031792:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576135.735256:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788576136.058626:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576136.758971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576137.082892:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788576137.591188:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788576137.591198:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788576137.591202:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788576137.591213:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8050 00000100:00080000:3.0:1788576137.591224:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788576137.591232:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788576137.591270:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2d80 x1875449616554368/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788576137.591301:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788576137.591305:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788576137.591306:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788576137.591310:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8050 00000100:00080000:3.0:1788576137.591314:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788576137.591318:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788576137.591327:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788576137.591339:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0a80 x1875449616554496/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788576137.591349:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788576137.591352:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788576137.591354:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788576137.591357:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8050 00000100:00080000:3.0:1788576137.591360:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788576137.591364:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788576137.591369:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788576137.591392:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2300 x1875449616554624/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788576137.591485:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616554368:192.168.202.113@tcp:250:kworker.0 00000800:00000100:2.0:1788576137.591548:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576137.591579:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576137/real 1788576137] req@ffff9ada604e2d80 x1875449616554368/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576162 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576137.591594:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2d80 x1875449616554368/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576162 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576137.591604:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576137.591607:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576137.591610:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616554368:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788576137.591618:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616554496:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576137.591639:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576137.591652:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576137/real 1788576137] req@ffff9ada604e0a80 x1875449616554496/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576162 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576137.591662:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0a80 x1875449616554496/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576162 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576137.591670:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576137.591673:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576137.591675:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616554496:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576137.591681:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616554624:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576137.591699:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576137.591709:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576137/real 1788576137] req@ffff9ada604e2300 x1875449616554624/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576162 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576137.591720:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2300 x1875449616554624/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576162 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576137.591745:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576137.591749:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576137.591752:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616554624:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576137.785661:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576138.104188:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576138.808001:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788576139.127071:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576139.831205:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576140.152719:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576140.861250:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576141.175072:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576141.879163:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788576142.199150:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788576142.714430:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788576142.714439:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788576142.714442:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788576142.714450:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8055 00000100:00080000:3.0:1788576142.714459:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788576142.714465:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788576142.714494:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0000 x1875449616554752/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788576142.714521:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788576142.714523:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788576142.714524:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788576142.714527:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8055 00000100:00080000:3.0:1788576142.714529:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788576142.714531:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788576142.714537:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788576142.714544:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1c00 x1875449616554880/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788576142.714550:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788576142.714552:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788576142.714553:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788576142.714556:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8055 00000100:00080000:3.0:1788576142.714558:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788576142.714559:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788576142.714561:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788576142.714566:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3b80 x1875449616555008/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788576142.714638:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616554752:192.168.202.113@tcp:250:kworker.0 00000800:00000100:2.0:1788576142.714684:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576142.714710:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576142/real 1788576142] req@ffff9ada604e0000 x1875449616554752/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576167 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576142.714722:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0000 x1875449616554752/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576167 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576142.714728:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576142.714731:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576142.714733:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616554752:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788576142.714738:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616554880:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576142.714749:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576142.714755:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576142/real 1788576142] req@ffff9ada604e1c00 x1875449616554880/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576167 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576142.714762:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1c00 x1875449616554880/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576167 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576142.714770:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576142.714772:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576142.714774:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616554880:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576142.714777:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616555008:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576142.714786:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576142.714791:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576142/real 1788576142] req@ffff9ada604e3b80 x1875449616555008/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576167 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576142.714796:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3b80 x1875449616555008/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576167 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576142.714801:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576142.714803:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576142.714804:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616555008:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576142.903054:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576143.223614:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576143.926972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576144.248286:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576144.950993:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788576145.272278:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576145.975203:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576146.295158:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576146.998972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576147.319681:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788576147.831196:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576147.831207:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788576147.831211:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576147.831222:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8060 00000100:00080000:1.0:1788576147.831232:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576147.831240:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788576147.831280:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e300 x1875449616555136/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788576147.831313:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576147.831317:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788576147.831318:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576147.831322:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8060 00000100:00080000:1.0:1788576147.831326:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576147.831329:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576147.831338:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576147.831350:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f480 x1875449616555264/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788576147.831359:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576147.831362:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788576147.831364:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576147.831367:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8060 00000100:00080000:1.0:1788576147.831370:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576147.831373:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576147.831377:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576147.831385:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ea00 x1875449616555392/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788576147.831481:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616555136:192.168.202.113@tcp:250:kworker.0 00000800:00000100:2.0:1788576147.831541:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576147.831574:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576147/real 1788576147] req@ffff9ada4566e300 x1875449616555136/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576172 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576147.831605:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e300 x1875449616555136/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576172 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576147.831616:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576147.831621:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576147.831633:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616555136:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788576147.831641:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616555264:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576147.831663:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576147.831676:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576147/real 1788576147] req@ffff9ada4566f480 x1875449616555264/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576172 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576147.831687:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f480 x1875449616555264/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576172 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576147.831696:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576147.831698:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576147.831701:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616555264:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576147.831707:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616555392:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576147.831727:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576147.831738:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576147/real 1788576147] req@ffff9ada4566ea00 x1875449616555392/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576172 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576147.831749:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ea00 x1875449616555392/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576172 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576147.831756:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576147.831758:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576147.831761:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616555392:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576148.023502:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788576148.343165:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576149.047082:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576149.367283:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576150.071126:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576150.391992:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576151.094985:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788576151.415194:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576152.119150:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576152.439967:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788576152.951070:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576152.951081:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788576152.951085:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576152.951094:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8065 00000100:00080000:1.0:1788576152.951105:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576152.951112:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788576152.951158:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d500 x1875449616555520/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788576152.951191:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576152.951195:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788576152.951197:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576152.951201:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8065 00000100:00080000:1.0:1788576152.951205:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576152.951209:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576152.951218:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576152.951232:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ca80 x1875449616555648/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788576152.951242:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616555520:192.168.202.113@tcp:250:kworker.0 00000100:00080000:1.0:1788576152.951245:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576152.951252:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788576152.951254:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576152.951258:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8065 00000100:00080000:1.0:1788576152.951261:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576152.951264:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576152.951268:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576152.951279:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d180 x1875449616555776/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:2.0:1788576152.951307:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576152.951544:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576152/real 1788576152] req@ffff9ada4566d500 x1875449616555520/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576177 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576152.951564:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d500 x1875449616555520/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576177 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576152.951575:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576152.951578:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576152.951582:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616555520:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788576152.951616:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616555648:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576152.951652:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576152.951664:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576152/real 1788576152] req@ffff9ada4566ca80 x1875449616555648/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576177 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576152.951676:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ca80 x1875449616555648/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576177 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576152.951684:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576152.951687:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576152.951690:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616555648:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576152.951696:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616555776:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576152.951715:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576152.951725:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576152/real 1788576152] req@ffff9ada4566d180 x1875449616555776/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576177 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576152.951735:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d180 x1875449616555776/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576177 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576152.951743:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576152.951746:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576152.951749:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616555776:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576153.142971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576153.463366:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576154.168612:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788576154.487644:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576155.191231:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576155.511111:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576156.215095:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576156.535995:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576157.239697:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788576157.564328:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788576158.071052:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788576158.071064:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788576158.071068:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788576158.071078:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8070 00000100:00080000:3.0:1788576158.071088:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788576158.071095:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788576158.071135:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1f80 x1875449616555904/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788576158.071175:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788576158.071179:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788576158.071181:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788576158.071184:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8070 00000100:00080000:3.0:1788576158.071188:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788576158.071192:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788576158.071202:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788576158.071214:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2680 x1875449616556032/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788576158.071225:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788576158.071228:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788576158.071229:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788576158.071233:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8070 00000100:00080000:3.0:1788576158.071236:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788576158.071240:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788576158.071244:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788576158.071254:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3100 x1875449616556160/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788576158.071349:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616555904:192.168.202.113@tcp:250:kworker.0 00000800:00000100:2.0:1788576158.071411:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576158.071456:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576158/real 1788576158] req@ffff9ada604e1f80 x1875449616555904/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576183 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576158.071474:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1f80 x1875449616555904/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576183 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576158.071484:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576158.071488:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576158.071492:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616555904:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788576158.071499:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616556032:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576158.071521:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576158.071541:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576158/real 1788576158] req@ffff9ada604e2680 x1875449616556032/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576183 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576158.071559:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2680 x1875449616556032/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576183 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576158.071568:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576158.071571:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576158.071574:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616556032:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576158.071580:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616556160:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576158.071603:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576158.071614:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576158/real 1788576158] req@ffff9ada604e3100 x1875449616556160/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576183 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576158.071625:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3100 x1875449616556160/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576183 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576158.071633:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576158.071636:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576158.071639:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616556160:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576158.263075:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576158.583088:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576159.287138:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576159.612906:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576160.311573:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788576160.631114:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576161.335068:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576161.655435:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576162.359477:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576162.679166:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788576163.191096:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576163.191106:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788576163.191111:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576163.191121:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8075 00000100:00080000:1.0:1788576163.191131:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576163.191138:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788576163.191184:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f100 x1875449616556288/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788576163.191210:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576163.191213:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788576163.191215:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576163.191218:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8075 00000100:00080000:1.0:1788576163.191221:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576163.191224:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576163.191233:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576163.191244:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c700 x1875449616556416/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788576163.191252:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576163.191256:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788576163.191258:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576163.191262:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8075 00000100:00080000:1.0:1788576163.191265:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576163.191270:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576163.191275:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576163.191283:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c380 x1875449616556544/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788576163.191307:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616556288:192.168.202.113@tcp:250:kworker.0 00000800:00000100:2.0:1788576163.191398:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576163.191444:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576163/real 1788576163] req@ffff9ada4566f100 x1875449616556288/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576188 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576163.191465:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f100 x1875449616556288/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576188 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576163.191484:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576163.191489:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576163.191493:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616556288:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788576163.191501:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616556416:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576163.191528:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576163.191542:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576163/real 1788576163] req@ffff9ada4566c700 x1875449616556416/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576188 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576163.191554:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c700 x1875449616556416/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576188 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576163.191563:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576163.191567:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576163.191570:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616556416:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576163.191621:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616556544:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576163.191645:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576163.191657:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576163/real 1788576163] req@ffff9ada4566c380 x1875449616556544/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576188 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576163.191668:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c380 x1875449616556544/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576188 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576163.191676:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576163.191680:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576163.191683:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616556544:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576163.382987:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788576163.703397:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576164.408015:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576164.727206:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576165.431048:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576165.751626:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576166.455018:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788576166.775345:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576167.479006:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576167.801236:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788576168.311956:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576168.311970:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788576168.311975:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576168.311987:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8080 00000100:00080000:1.0:1788576168.312000:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576168.312008:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788576168.312057:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ce00 x1875449616556672/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788576168.312152:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616556672:192.168.202.113@tcp:250:kworker.0 00000800:00000100:2.0:1788576168.312263:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00080000:1.0:1788576168.312302:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576168.312306:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788576168.312308:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576168.312313:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8080 00000100:00080000:1.0:1788576168.312317:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576168.312321:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576168.312334:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576168.312348:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d880 x1875449616556800/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788576168.317403:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576168/real 1788576168] req@ffff9ada4566ce00 x1875449616556672/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576193 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576168.317443:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ce00 x1875449616556672/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576193 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576168.317456:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576168.317461:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576168.317466:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616556672:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788576168.317509:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616556800:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576168.321213:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576168.321611:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576168/real 1788576168] req@ffff9ada4566d880 x1875449616556800/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576193 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576168.321720:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d880 x1875449616556800/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576193 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576168.321735:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576168.321740:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576168.321744:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616556800:192.168.202.113@tcp:38:kworker.0 00000100:00080000:1.0:1788576168.322619:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576168.322640:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788576168.322645:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576168.322655:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8080 00000100:00080000:1.0:1788576168.322669:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576168.322687:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576168.322705:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576168.322753:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e680 x1875449616556928/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576168.324337:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616556928:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576168.324422:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576168.324450:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576168/real 1788576168] req@ffff9ada4566e680 x1875449616556928/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576193 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576168.324468:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e680 x1875449616556928/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576193 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576168.324479:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576168.324483:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576168.324486:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616556928:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576168.504304:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576168.823603:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576169.527233:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788576169.847110:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576170.552851:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576170.871041:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576171.575233:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576171.895317:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576172.598968:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788576172.920727:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576173.431103:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576173.431113:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576173.431117:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576173.431127:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8085 00000100:00080000:0.0:1788576173.431146:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576173.431155:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576173.431193:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616557056/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576173.431223:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576173.431227:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576173.431230:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576173.431234:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8085 00000100:00080000:0.0:1788576173.431237:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576173.431241:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576173.431250:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576173.431263:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d180 x1875449616557184/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576173.431272:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576173.431275:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576173.431277:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576173.431280:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8085 00000100:00080000:0.0:1788576173.431284:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576173.431288:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576173.431293:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576173.431302:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d500 x1875449616557312/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576173.431590:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616557056:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576173.431725:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576173.431770:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576173/real 1788576173] req@ffff9ada4557e680 x1875449616557056/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576198 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576173.431788:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616557056/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576198 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576173.431798:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576173.431802:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576173.431806:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616557056:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576173.431813:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616557184:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576173.431834:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576173.431846:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576173/real 1788576173] req@ffff9ada4557d180 x1875449616557184/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576198 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576173.431886:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d180 x1875449616557184/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576198 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576173.431894:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576173.431898:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576173.431901:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616557184:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576173.431906:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616557312:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576173.431924:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576173.431935:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576173/real 1788576173] req@ffff9ada4557d500 x1875449616557312/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576198 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576173.431946:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d500 x1875449616557312/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576198 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576173.431954:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576173.431957:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576173.431961:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616557312:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576173.623076:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576173.948747:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576174.650036:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576174.970873:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576175.672197:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788576175.991151:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576176.695155:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576177.015079:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576177.719978:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576178.039071:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576178.551194:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576178.551204:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576178.551209:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576178.551220:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8091 00000100:00080000:0.0:1788576178.551231:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576178.551238:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576178.551274:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616557440/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576178.551308:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576178.551312:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576178.551314:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576178.551318:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8091 00000100:00080000:0.0:1788576178.551322:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576178.551326:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576178.551335:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576178.551347:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557df80 x1875449616557568/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576178.551357:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576178.551361:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576178.551362:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576178.551366:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8091 00000100:00080000:0.0:1788576178.551369:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576178.551373:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576178.551377:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576178.551389:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d500 x1875449616557696/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576178.551473:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616557440:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576178.551563:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616557568:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576178.551590:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616557696:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576178.742975:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788576179.063185:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576179.766988:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576180.087122:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576180.793328:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576181.111091:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1788576181.623468:0:2373:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:1.0:1788576181.623989:0:2373:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1788576181.624005:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:1.0:1788576181.624037:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:1.0:1788576181.624062:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:0.0:1788576181.625714:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576178/real 1788576181] req@ffff9ada4557ea00 x1875449616557440/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576203 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788576181.625744:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616557440/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576203 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788576181.625765:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788576181.625769:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788576181.625773:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616557440:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788576181.625781:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576178/real 1788576181] req@ffff9ada4557df80 x1875449616557568/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576203 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788576181.625793:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557df80 x1875449616557568/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576203 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788576181.625800:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788576181.625803:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788576181.625807:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616557568:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788576181.625813:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576178/real 1788576181] req@ffff9ada4557d500 x1875449616557696/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576203 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788576181.625823:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d500 x1875449616557696/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576203 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788576181.625829:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788576181.625833:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788576181.625835:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616557696:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576181.814983:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788576182.135195:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576182.838972:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788576183.159486:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576183.670935:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576183.670945:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576183.670950:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576183.670959:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8096 00000100:00080000:0.0:1788576183.670968:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576183.670975:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576183.671008:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d500 x1875449616557824/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576183.671061:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576183.671065:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576183.671067:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576183.671070:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8096 00000100:00080000:0.0:1788576183.671074:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576183.671077:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576183.671085:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576183.671096:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557df80 x1875449616557952/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576183.671105:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576183.671108:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576183.671110:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576183.671113:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8096 00000100:00080000:0.0:1788576183.671115:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576183.671118:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576183.671122:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576183.671130:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616558080/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576183.671217:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616557824:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576183.671271:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576183.671302:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576183/real 1788576183] req@ffff9ada4557d500 x1875449616557824/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576208 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576183.671318:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d500 x1875449616557824/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576208 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576183.671328:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576183.671332:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576183.671335:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616557824:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576183.671341:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616557952:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576183.671361:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576183.671371:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576183/real 1788576183] req@ffff9ada4557df80 x1875449616557952/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576208 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576183.671382:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557df80 x1875449616557952/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576208 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576183.671391:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576183.671393:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576183.671396:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616557952:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576183.671401:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616558080:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576183.671416:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576183.671424:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576183/real 1788576183] req@ffff9ada4557ea00 x1875449616558080/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576208 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576183.671434:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616558080/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576208 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576183.671442:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576183.671445:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576183.671448:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616558080:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576183.862971:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576184.184940:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576184.887023:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788576185.213344:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576185.911138:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788576186.231060:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576186.935028:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576187.255273:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576187.959034:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788576188.279106:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788576188.791072:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788576188.791082:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788576188.791086:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788576188.791097:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8101 00000100:00080000:3.0:1788576188.791109:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788576188.791118:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788576188.791164:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2a00 x1875449616558208/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788576188.791220:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616558208:192.168.202.113@tcp:250:kworker.0 00000800:00000100:0.0:1788576188.791299:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00080000:3.0:1788576188.791328:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788576188.791331:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788576188.791332:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788576188.791335:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8101 00000100:00080000:3.0:1788576188.791338:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788576188.791340:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788576188.791350:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788576188.791360:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1500 x1875449616558336/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788576188.791578:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576188/real 1788576188] req@ffff9ada604e2a00 x1875449616558208/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576213 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788576188.791593:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2a00 x1875449616558208/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576213 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788576188.791602:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788576188.791605:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788576188.791608:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616558208:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788576188.791630:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616558336:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788576188.791648:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788576188.791657:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576188/real 1788576188] req@ffff9ada604e1500 x1875449616558336/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576213 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788576188.791664:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1500 x1875449616558336/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576213 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788576188.791669:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788576188.791671:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788576188.791674:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616558336:192.168.202.113@tcp:38:kworker.0 00000100:00080000:3.0:1788576188.791807:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788576188.791810:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788576188.791812:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788576188.791815:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8101 00000100:00080000:3.0:1788576188.791818:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788576188.791822:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788576188.791826:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788576188.791836:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0e00 x1875449616558464/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788576188.791869:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616558464:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788576188.791888:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788576188.791960:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576188/real 1788576188] req@ffff9ada604e0e00 x1875449616558464/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576213 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788576188.791977:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0e00 x1875449616558464/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576213 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788576188.791983:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788576188.791986:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788576188.791988:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616558464:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576188.983157:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788576189.303093:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576190.008400:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576190.327126:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576191.031648:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788576191.351107:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576192.055015:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788576192.374980:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576193.078990:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576193.399103:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788576193.910993:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788576193.911003:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788576193.911007:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788576193.911017:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8106 00000100:00080000:3.0:1788576193.911026:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788576193.911033:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788576193.911069:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0e00 x1875449616558592/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788576193.911149:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616558592:192.168.202.113@tcp:250:kworker.0 00000800:00000100:0.0:1788576193.911228:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00080000:3.0:1788576193.911452:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788576193.911456:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788576193.911458:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788576193.911463:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8106 00000100:00080000:3.0:1788576193.911467:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788576193.911471:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788576193.911480:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788576193.911493:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1500 x1875449616558720/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788576193.911689:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576193/real 1788576193] req@ffff9ada604e0e00 x1875449616558592/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576218 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788576193.911707:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0e00 x1875449616558592/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576218 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788576193.911718:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788576193.911722:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788576193.911726:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616558592:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788576193.911749:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616558720:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788576193.911772:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788576193.911785:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576193/real 1788576193] req@ffff9ada604e1500 x1875449616558720/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576218 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788576193.911802:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1500 x1875449616558720/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576218 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788576193.911811:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788576193.911813:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788576193.911816:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616558720:192.168.202.113@tcp:38:kworker.0 00000100:00080000:3.0:1788576193.914057:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788576193.914066:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788576193.914069:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788576193.914077:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8106 00000100:00080000:3.0:1788576193.914086:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788576193.914091:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788576193.914102:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788576193.914130:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2a00 x1875449616558848/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788576193.914651:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616558848:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788576193.914704:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788576193.914726:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576193/real 1788576193] req@ffff9ada604e2a00 x1875449616558848/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576218 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788576193.914741:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2a00 x1875449616558848/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576218 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788576193.914751:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788576193.914755:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788576193.914759:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616558848:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576194.109327:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788576194.423727:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576195.127199:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788576195.447923:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576196.151700:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576196.471097:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576197.175018:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788576197.495339:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576198.205966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788576198.519079:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576199.031125:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576199.031139:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576199.031145:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576199.031158:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8111 00000100:00080000:0.0:1788576199.031172:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576199.031181:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576199.031346:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d180 x1875449616558976/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576199.031420:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616558976:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576199.031499:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00080000:0.0:1788576199.031536:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576199.031541:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576199.031543:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576199.031548:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8111 00000100:00080000:0.0:1788576199.031552:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576199.031557:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576199.031568:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576199.031583:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616559104/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576199.031735:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576199/real 1788576199] req@ffff9ada4557d180 x1875449616558976/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576224 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576199.031757:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d180 x1875449616558976/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576224 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576199.031769:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576199.031773:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576199.031778:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616558976:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576199.031802:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616559104:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576199.031926:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576199.031940:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576199/real 1788576199] req@ffff9ada4557e680 x1875449616559104/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576224 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576199.032017:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616559104/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576224 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576199.032024:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576199.032027:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576199.032031:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616559104:192.168.202.113@tcp:38:kworker.0 00000100:00080000:0.0:1788576199.032208:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576199.032212:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576199.032214:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576199.032219:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8111 00000100:00080000:0.0:1788576199.032223:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576199.032227:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576199.032234:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576199.032248:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616559232/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576199.032304:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616559232:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576199.032327:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576199.032339:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576199/real 1788576199] req@ffff9ada4557e300 x1875449616559232/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576224 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576199.032350:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616559232/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576224 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576199.032357:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576199.032360:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576199.032363:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616559232:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576199.223320:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576199.543789:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576200.247014:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:0.0:1788576200.567762:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576201.271069:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:2.0:1788576201.591264:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576202.295009:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576202.618968:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576203.319012:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576203.639201:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576204.151031:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576204.151045:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576204.151051:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576204.151063:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8116 00000100:00080000:0.0:1788576204.151075:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576204.151084:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576204.151136:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616559360/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576204.151175:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576204.151179:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576204.151181:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576204.151185:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8116 00000100:00080000:0.0:1788576204.151189:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576204.151193:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576204.151205:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576204.151217:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616559488/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576204.151226:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576204.151229:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576204.151231:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576204.151234:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8116 00000100:00080000:0.0:1788576204.151237:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576204.151241:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576204.151245:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576204.151255:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616559616/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576204.151411:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616559360:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576204.151492:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576204.151534:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576204/real 1788576204] req@ffff9ada4557ed80 x1875449616559360/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576229 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576204.151554:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616559360/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576229 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576204.151566:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576204.151571:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576204.151575:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616559360:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576204.151583:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616559488:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576204.151607:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576204.151621:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576204/real 1788576204] req@ffff9ada4557fb80 x1875449616559488/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576229 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576204.151633:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616559488/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576229 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576204.151641:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576204.151644:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576204.151647:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616559488:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576204.151653:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616559616:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576204.151671:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576204.151681:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576204/real 1788576204] req@ffff9ada4557f800 x1875449616559616/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576229 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576204.151692:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616559616/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576229 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576204.151699:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576204.151702:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576204.151706:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616559616:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576204.407969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576204.664987:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576205.431954:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576205.687568:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576206.455247:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576206.712899:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576207.479128:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576207.735178:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576208.503493:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576208.760268:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788576209.273785:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576209.273794:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788576209.273798:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576209.273807:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8121 00000100:00080000:2.0:1788576209.273817:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576209.273824:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788576209.273882:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616559744/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576209.273911:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576209.273915:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788576209.273917:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576209.273920:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8121 00000100:00080000:2.0:1788576209.273924:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576209.273928:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576209.273937:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576209.273950:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616559872/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576209.273961:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576209.273965:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788576209.273967:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576209.273972:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8121 00000100:00080000:2.0:1788576209.273975:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576209.273979:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576209.273983:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576209.273992:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616560000/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576209.274020:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616559744:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576209.274289:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576209.274335:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576209/real 1788576209] req@ffff9ada60624e00 x1875449616559744/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576234 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576209.274357:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616559744/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576234 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576209.274663:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576209.274670:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576209.274674:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616559744:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576209.274683:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616559872:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576209.274710:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576209.274724:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576209/real 1788576209] req@ffff9ada60627480 x1875449616559872/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576234 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576209.274737:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616559872/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576234 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576209.274763:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576209.274767:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576209.274770:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616559872:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576209.274803:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616560000:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576209.274822:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576209.274833:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576209/real 1788576209] req@ffff9ada60625f80 x1875449616560000/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576234 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576209.274844:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616560000/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576234 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576209.274888:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576209.274892:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576209.274896:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616560000:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576209.527294:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576209.783528:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576210.550977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576210.807229:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576211.574984:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576211.834496:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576212.599373:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576212.855219:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576213.622969:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576213.879210:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788576214.391979:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788576214.391991:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788576214.391996:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788576214.392009:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8126 00000100:00080000:3.0:1788576214.392023:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788576214.392031:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788576214.392090:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3100 x1875449616560128/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788576214.392134:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788576214.392138:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788576214.392139:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788576214.392143:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8126 00000100:00080000:3.0:1788576214.392147:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788576214.392152:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788576214.392164:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788576214.392177:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2680 x1875449616560256/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788576214.392187:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788576214.392190:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788576214.392192:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788576214.392196:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8126 00000100:00080000:3.0:1788576214.392199:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788576214.392203:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788576214.392207:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788576214.392217:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1f80 x1875449616560384/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788576214.392363:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616560128:192.168.202.113@tcp:250:kworker.0 00000800:00000100:0.0:1788576214.392524:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788576214.392574:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576214/real 1788576214] req@ffff9ada604e3100 x1875449616560128/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576239 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788576214.392595:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3100 x1875449616560128/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576239 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788576214.392607:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788576214.392611:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788576214.392615:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616560128:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788576214.392622:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616560256:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788576214.392649:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788576214.392662:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576214/real 1788576214] req@ffff9ada604e2680 x1875449616560256/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576239 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788576214.392675:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2680 x1875449616560256/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576239 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788576214.392683:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788576214.392687:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788576214.392699:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616560256:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788576214.392706:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616560384:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788576214.392726:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788576214.392738:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576214/real 1788576214] req@ffff9ada604e1f80 x1875449616560384/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576239 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788576214.392749:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1f80 x1875449616560384/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576239 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788576214.392757:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788576214.392760:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788576214.392763:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616560384:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576214.647044:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576214.905020:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576215.671022:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576215.928571:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576216.695990:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576216.954251:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576217.719038:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576217.975197:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576218.743006:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576219.000322:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788576219.511076:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576219.511091:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788576219.511097:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576219.511110:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8132 00000100:00080000:1.0:1788576219.511123:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576219.511132:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788576219.511194:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e680 x1875449616560512/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788576219.511231:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576219.511236:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788576219.511245:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576219.511250:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8132 00000100:00080000:1.0:1788576219.511254:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576219.511258:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576219.511269:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576219.511283:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d880 x1875449616560640/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788576219.511293:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576219.511297:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788576219.511299:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576219.511304:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8132 00000100:00080000:1.0:1788576219.511307:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576219.511310:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576219.511314:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576219.511332:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ce00 x1875449616560768/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788576219.511437:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616560512:192.168.202.113@tcp:250:kworker.0 00000800:00000100:0.0:1788576219.511526:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788576219.511570:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576219/real 1788576219] req@ffff9ada4566e680 x1875449616560512/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576244 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788576219.511589:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e680 x1875449616560512/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576244 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788576219.511601:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788576219.511606:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788576219.511610:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616560512:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788576219.511617:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616560640:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788576219.511668:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788576219.511681:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576219/real 1788576219] req@ffff9ada4566d880 x1875449616560640/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576244 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788576219.511694:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d880 x1875449616560640/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576244 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788576219.511709:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788576219.511712:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788576219.511715:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616560640:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788576219.511721:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616560768:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788576219.511741:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788576219.511751:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576219/real 1788576219] req@ffff9ada4566ce00 x1875449616560768/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576244 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788576219.511762:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ce00 x1875449616560768/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576244 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788576219.511770:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788576219.511773:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788576219.511777:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616560768:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576219.767153:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576220.023072:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576220.791019:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576221.047680:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576221.829598:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576222.071145:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576222.839059:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576223.095742:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576223.863019:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576224.119607:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576224.631497:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576224.631511:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576224.631516:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576224.631529:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8137 00000100:00080000:0.0:1788576224.631540:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576224.631547:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576224.631611:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616560896/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576224.631645:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576224.631650:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576224.631652:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576224.631656:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8137 00000100:00080000:0.0:1788576224.631659:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576224.631663:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576224.631672:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576224.631683:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616561024/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576224.631693:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576224.631697:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576224.631699:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576224.631702:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8137 00000100:00080000:0.0:1788576224.631705:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576224.631708:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576224.631712:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576224.631721:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616561152/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576224.634426:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616560896:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576224.634504:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576224.634542:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576224/real 1788576224] req@ffff9ada4557dc00 x1875449616560896/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576249 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576224.634561:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616560896/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576249 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576224.634573:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576224.634577:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576224.634592:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616560896:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576224.634600:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616561024:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576224.634624:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576224.634636:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576224/real 1788576224] req@ffff9ada4557d880 x1875449616561024/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576249 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576224.634646:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616561024/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576249 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576224.634653:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576224.634656:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576224.634658:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616561024:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576224.634667:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616561152:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576224.634686:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576224.634696:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576224/real 1788576224] req@ffff9ada4557ca80 x1875449616561152/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576249 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576224.634706:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616561152/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576249 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576224.634713:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576224.634716:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576224.634720:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616561152:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576224.887243:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576225.145197:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576225.910975:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576226.167069:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576226.935014:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576227.192473:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576227.959795:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576228.216070:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576228.983611:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576229.239605:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576229.751074:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576229.751083:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576229.751087:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576229.751097:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8142 00000100:00080000:0.0:1788576229.751108:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576229.751116:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576229.751163:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616561280/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576229.751197:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576229.751200:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576229.751201:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576229.751206:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8142 00000100:00080000:0.0:1788576229.751210:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576229.751214:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576229.751222:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576229.751235:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ce00 x1875449616561408/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576229.751243:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576229.751246:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576229.751248:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576229.751251:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8142 00000100:00080000:0.0:1788576229.751255:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576229.751258:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576229.751262:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576229.751271:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616561536/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576229.751427:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616561280:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576229.751485:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576229.751515:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576229/real 1788576229] req@ffff9ada4557c700 x1875449616561280/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576254 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576229.751560:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616561280/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576254 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576229.751570:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576229.751573:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576229.751577:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616561280:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576229.751583:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616561408:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576229.751606:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576229.751617:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576229/real 1788576229] req@ffff9ada4557ce00 x1875449616561408/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576254 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576229.751628:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ce00 x1875449616561408/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576254 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576229.751634:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576229.751637:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576229.751639:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616561408:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576229.751645:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616561536:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576229.751661:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576229.751671:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576229/real 1788576229] req@ffff9ada4557f480 x1875449616561536/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576254 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576229.751680:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616561536/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576254 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576229.751688:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576229.751690:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576229.751693:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616561536:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576230.007031:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576230.266288:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576231.030989:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576231.289005:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576232.054992:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576232.312330:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576233.082723:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576233.335118:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576234.103020:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576234.359066:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788576234.871026:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576234.871038:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788576234.871043:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576234.871054:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8147 00000100:00080000:2.0:1788576234.871065:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576234.871074:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788576234.871112:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616561664/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576234.871141:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576234.871145:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788576234.871147:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576234.871151:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8147 00000100:00080000:2.0:1788576234.871154:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576234.871158:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576234.871168:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576234.871181:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616561792/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576234.871189:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576234.871192:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788576234.871194:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576234.871197:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8147 00000100:00080000:2.0:1788576234.871200:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576234.871203:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576234.871206:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576234.871215:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616561920/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576234.871297:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616561664:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576234.871357:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576234.871391:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576234/real 1788576234] req@ffff9ada60625180 x1875449616561664/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576259 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576234.871409:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616561664/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576259 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576234.871418:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576234.871422:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576234.871425:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616561664:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576234.871432:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616561792:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576234.871454:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576234.871477:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576234/real 1788576234] req@ffff9ada60626d80 x1875449616561792/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576259 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576234.871489:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616561792/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576259 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576234.871497:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576234.871500:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576234.871503:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616561792:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576234.871509:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616561920:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576234.871529:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576234.871539:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576234/real 1788576234] req@ffff9ada60627b80 x1875449616561920/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576259 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576234.871550:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616561920/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576259 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576234.871562:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576234.871564:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576234.871567:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616561920:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576235.128115:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576235.383360:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576236.151326:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576236.407140:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576237.175029:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576237.431055:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576238.199150:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576238.455040:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576239.222996:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576239.479081:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788576239.991503:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576239.991513:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788576239.991516:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576239.991526:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8152 00000100:00080000:2.0:1788576239.991534:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576239.991541:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788576239.991578:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626300 x1875449616562048/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576239.991611:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576239.991615:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788576239.991617:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576239.991621:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8152 00000100:00080000:2.0:1788576239.991625:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576239.991629:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576239.991637:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576239.991649:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627800 x1875449616562176/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576239.991657:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576239.991660:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788576239.991663:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576239.991666:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8152 00000100:00080000:2.0:1788576239.991669:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576239.991672:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576239.991676:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576239.991686:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616562304/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576239.992263:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616562048:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576239.992338:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576239.992372:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576239/real 1788576239] req@ffff9ada60626300 x1875449616562048/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576264 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576239.992418:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626300 x1875449616562048/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576264 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576239.992429:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576239.992433:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576239.992437:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616562048:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576239.992444:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616562176:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576239.992467:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576239.992479:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576239/real 1788576239] req@ffff9ada60627800 x1875449616562176/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576264 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576239.992491:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627800 x1875449616562176/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576264 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576239.992499:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576239.992502:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576239.992505:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616562176:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576239.992511:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616562304:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576239.992529:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576239.992551:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576239/real 1788576239] req@ffff9ada60627b80 x1875449616562304/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576264 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576239.992564:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616562304/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576264 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576239.992570:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576239.992574:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576239.992577:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616562304:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576240.247455:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576240.505809:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576241.270982:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576241.527176:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576242.294981:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576242.552217:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576243.319010:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576243.576957:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576244.343013:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576244.600701:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576245.111233:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576245.111245:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576245.111250:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576245.111261:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8157 00000100:00080000:0.0:1788576245.111272:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576245.111281:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576245.111338:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c000 x1875449616562432/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576245.111426:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616562432:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788576245.112199:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576245.112205:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576245.112209:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576245.112215:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8157 00000100:00080000:0.0:1788576245.112220:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576245.112226:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576245.112239:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576245.112260:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c380 x1875449616562560/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788576245.153970:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616562560:192.168.202.113@tcp:38:kworker.0 00000100:00080000:0.0:1788576245.158711:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576245.158723:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576245.158727:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576245.158738:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8157 00000100:00080000:0.0:1788576245.158748:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576245.158755:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576245.158801:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576245.158843:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616562688/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788576245.159735:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616562688:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576245.367118:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576245.626141:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576246.391380:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576246.647408:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576247.414975:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576247.671379:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1788576248.190164:0:2371:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:1.0:1788576248.190224:0:2371:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1788576248.190243:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:1.0:1788576248.190288:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:1.0:1788576248.190300:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:2.0:1788576248.192937:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576245/real 1788576248] req@ffff9ada4557f480 x1875449616562688/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576270 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576248.192972:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616562688/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576270 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576248.192985:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576248.192990:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576248.192994:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616562688:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576248.193004:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576245/real 1788576248] req@ffff9ada4557c380 x1875449616562560/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576270 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576248.193016:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c380 x1875449616562560/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576270 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576248.193025:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576248.193028:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576248.193031:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616562560:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576248.193038:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576245/real 1788576248] req@ffff9ada4557c000 x1875449616562432/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576270 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576248.193051:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c000 x1875449616562432/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576270 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576248.193057:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576248.193060:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576248.193063:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616562432:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788576248.438993:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576248.704365:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576249.464674:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576249.720876:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788576250.234984:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576250.234994:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788576250.234998:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576250.235008:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8162 00000100:00080000:2.0:1788576250.235017:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576250.235025:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788576250.235056:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616562816/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576250.235088:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576250.235092:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788576250.235093:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576250.235097:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8162 00000100:00080000:2.0:1788576250.235100:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576250.235103:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576250.235112:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576250.235135:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616562944/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576250.235153:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576250.235157:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788576250.235158:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576250.235163:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8162 00000100:00080000:2.0:1788576250.235168:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576250.235172:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576250.235177:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576250.235192:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616563072/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576250.235196:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616562816:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576250.235314:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576250.235379:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576250/real 1788576250] req@ffff9ada60626d80 x1875449616562816/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576275 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576250.235401:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616562816/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576275 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576250.235426:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576250.235431:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576250.235436:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616562816:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576250.235444:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616562944:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576250.235471:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576250.235485:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576250/real 1788576250] req@ffff9ada60625180 x1875449616562944/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576275 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576250.235497:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616562944/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576275 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576250.235504:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576250.235507:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576250.235510:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616562944:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576250.235541:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616563072:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576250.235561:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576250.235571:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576250/real 1788576250] req@ffff9ada60625f80 x1875449616563072/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576275 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576250.235596:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616563072/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576275 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576250.235605:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576250.235609:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576250.235612:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616563072:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576250.488742:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576250.743168:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576251.511024:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576251.779526:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576252.535342:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576252.793318:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576253.558990:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576253.817223:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576254.584668:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576254.840496:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788576255.351135:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576255.351175:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788576255.351182:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576255.351196:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8167 00000100:00080000:2.0:1788576255.351211:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576255.351222:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788576255.351273:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616563200/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576255.351313:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576255.351318:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788576255.351320:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576255.351324:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8167 00000100:00080000:2.0:1788576255.351328:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576255.351332:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576255.351344:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576255.351356:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616563328/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576255.351485:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576255.351489:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788576255.351491:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576255.351495:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8167 00000100:00080000:2.0:1788576255.351498:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576255.351502:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576255.351509:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576255.351523:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616563456/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576255.351640:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616563200:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576255.352003:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576255.352225:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576255/real 1788576255] req@ffff9ada60627480 x1875449616563200/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576280 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576255.352247:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616563200/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576280 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576255.352258:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576255.352263:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576255.352266:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616563200:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576255.352274:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616563328:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576255.352301:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576255.352314:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576255/real 1788576255] req@ffff9ada60624e00 x1875449616563328/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576280 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576255.352326:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616563328/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576280 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576255.352334:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576255.352337:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576255.352341:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616563328:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576255.352347:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616563456:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576255.353527:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576255.353567:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576255/real 1788576255] req@ffff9ada60625c00 x1875449616563456/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576280 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576255.353587:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616563456/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576280 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576255.353604:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576255.353609:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576255.353613:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616563456:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576255.608094:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576255.865051:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576256.630999:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576256.887454:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576257.656462:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576257.911547:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576258.897065:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576258.937738:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576259.959430:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576259.959610:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788576260.471010:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576260.471024:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788576260.471029:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576260.471043:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8173 00000100:00080000:2.0:1788576260.471056:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576260.471065:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788576260.471238:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616563584/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576260.471276:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576260.471280:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788576260.471282:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576260.471286:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8173 00000100:00080000:2.0:1788576260.471291:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576260.471295:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788576260.471406:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616563584:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576260.471490:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576260.471539:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576260/real 1788576260] req@ffff9ada60624380 x1875449616563584/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576285 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576260.471560:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616563584/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576285 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576260.471572:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576260.471576:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576260.471580:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616563584:192.168.202.113@tcp:250:kworker.0 00000008:00100000:2.0:1788576260.471651:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576260.471665:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616563712/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576260.471685:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576260.471689:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788576260.471692:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576260.471696:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8173 00000100:00080000:2.0:1788576260.471700:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576260.471704:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576260.471710:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576260.471720:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616563840/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576260.471888:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616563712:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576260.471915:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576260.471928:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576260/real 1788576260] req@ffff9ada60625880 x1875449616563712/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576285 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576260.471941:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616563712/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576285 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576260.471950:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576260.471954:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576260.471957:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616563712:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576260.471963:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616563840:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576260.471980:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576260.471990:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576260/real 1788576260] req@ffff9ada60624a80 x1875449616563840/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576285 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576260.472002:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616563840/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576285 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576260.472010:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576260.472014:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576260.472017:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616563840:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576260.983044:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576260.983206:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576262.007017:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576262.007119:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576263.031006:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576263.031209:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576264.054970:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576264.055699:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576265.078980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576265.079199:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576265.596522:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576265.596540:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576265.596550:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576265.596560:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8178 00000100:00080000:0.0:1788576265.596582:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576265.596595:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576265.596639:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ce00 x1875449616563968/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576265.596810:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616563968:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576265.596907:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00080000:0.0:1788576265.596947:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576265.596962:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576265.596965:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576265.596969:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8178 00000100:00080000:0.0:1788576265.596974:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576265.596979:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576265.596988:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576265.597014:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616564096/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576265.599265:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576265/real 1788576265] req@ffff9ada4557ce00 x1875449616563968/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576290 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576265.599313:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ce00 x1875449616563968/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576290 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576265.599325:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576265.599330:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576265.599334:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616563968:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576265.599374:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616564096:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576265.599431:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576265.599447:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576265/real 1788576265] req@ffff9ada4557c700 x1875449616564096/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576290 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576265.599459:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616564096/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576290 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576265.599468:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576265.599471:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576265.599474:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616564096:192.168.202.113@tcp:38:kworker.0 00000100:00080000:0.0:1788576265.600376:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576265.600382:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576265.600385:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576265.600392:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8178 00000100:00080000:0.0:1788576265.600398:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576265.600405:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576265.600415:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576265.600439:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616564224/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576265.600589:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616564224:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576265.600622:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576265.600638:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576265/real 1788576265] req@ffff9ada4557ca80 x1875449616564224/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576290 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576265.600717:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616564224/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576290 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576265.600727:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576265.600730:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576265.600733:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616564224:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576266.102992:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576266.104236:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788576267.127461:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576267.127519:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00000100:00100000:2.0:1788576268.151124:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576268.152113:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576269.176895:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576269.178010:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576270.491996:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576270.492813:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576270.711241:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576270.711251:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576270.711255:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576270.711265:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8183 00000100:00080000:0.0:1788576270.711274:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576270.711281:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576270.711316:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616564352/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576270.711804:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616564352:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788576270.712238:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576270.712242:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576270.712244:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576270.712249:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8183 00000100:00080000:0.0:1788576270.712253:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576270.712257:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576270.712266:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576270.712281:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616564480/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576270.712293:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576270.712296:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576270.712297:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576270.712300:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8183 00000100:00080000:0.0:1788576270.712303:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576270.712307:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576270.712312:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576270.712319:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616564608/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1788576270.712640:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576270.712787:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576270/real 1788576270] req@ffff9ada4557d880 x1875449616564352/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576295 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576270.712804:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616564352/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576295 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576270.712814:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576270.712817:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576270.712820:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616564352:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576270.713633:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616564480:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576270.714745:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576270.714768:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576270/real 1788576270] req@ffff9ada4557dc00 x1875449616564480/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576295 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576270.714784:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616564480/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576295 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576270.714794:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576270.714797:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576270.714801:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616564480:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576270.714809:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616564608:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576270.714829:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576270.714838:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576270/real 1788576270] req@ffff9ada4557f800 x1875449616564608/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576295 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576270.714849:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616564608/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576295 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576270.714872:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576270.714876:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576270.714879:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616564608:192.168.202.113@tcp:38:kworker.0 00010000:00010000:1.0:1788576271.544412:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576271.546572:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00000100:00100000:3.0:1788576272.567722:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576272.571782:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576273.595312:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576273.612387:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788576274.615996:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576274.617354:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00000100:00100000:3.0:1788576275.641212:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576275.645521:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788576275.831270:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576275.831279:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788576275.831283:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576275.831292:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8188 00000100:00080000:2.0:1788576275.831302:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576275.831310:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788576275.831345:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616564736/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576275.831377:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576275.831381:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788576275.831382:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576275.831386:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8188 00000100:00080000:2.0:1788576275.831389:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576275.831392:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576275.831401:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576275.831413:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616564864/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576275.831421:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576275.831425:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788576275.831426:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576275.831429:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8188 00000100:00080000:2.0:1788576275.831432:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576275.831436:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576275.831439:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576275.831448:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616564992/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576275.832590:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616564736:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576275.832653:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576275.832688:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576275/real 1788576275] req@ffff9ada60625500 x1875449616564736/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576300 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576275.832705:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616564736/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576300 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576275.832716:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576275.832720:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576275.832724:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616564736:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576275.832733:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616564864:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576275.832753:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576275.832780:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576275/real 1788576275] req@ffff9ada60626a00 x1875449616564864/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576300 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576275.832793:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616564864/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576300 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576275.832801:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576275.833019:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576275.833023:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616564864:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576275.833045:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616564992:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576275.833069:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576275.833082:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576275/real 1788576275] req@ffff9ada60626680 x1875449616564992/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576300 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576275.833093:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616564992/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576300 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576275.833100:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576275.833103:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576275.833106:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616564992:192.168.202.113@tcp:38:kworker.0 00010000:00010000:3.0:1788576276.664013:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576276.664370:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00000100:00100000:2.0:1788576277.686932:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576277.694589:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576278.721635:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576278.721912:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788576279.740027:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576279.740152:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00000100:00100000:2.0:1788576280.762696:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576280.762918:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576280.951073:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576280.951085:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576280.951089:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576280.951098:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8193 00000100:00080000:0.0:1788576280.951108:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576280.951117:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576280.951166:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616565120/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576280.951201:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576280.951204:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576280.951206:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576280.951211:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8193 00000100:00080000:0.0:1788576280.951214:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576280.951227:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576280.951236:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576280.951248:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616565248/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576280.951257:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576280.951260:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576280.951261:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576280.951265:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8193 00000100:00080000:0.0:1788576280.951269:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576280.951272:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576280.951276:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576280.951284:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616565376/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576280.951631:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616565120:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576280.951698:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576280.951732:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576280/real 1788576280] req@ffff9ada4557fb80 x1875449616565120/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576305 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576280.951749:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616565120/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576305 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576280.951770:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576280.951774:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576280.951778:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616565120:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576280.951785:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616565248:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576280.951806:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576280.951818:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576280/real 1788576280] req@ffff9ada4557ed80 x1875449616565248/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576305 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576280.951828:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616565248/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576305 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576280.951835:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576280.951838:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576280.951893:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616565248:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576280.951901:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616565376:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576280.951921:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576280.951932:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576280/real 1788576280] req@ffff9ada4557e300 x1875449616565376/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576305 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576280.951947:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616565376/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576305 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576280.951954:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576280.951956:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576280.951959:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616565376:192.168.202.113@tcp:38:kworker.0 00010000:00010000:1.0:1788576281.784974:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576281.787708:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00000100:00100000:3.0:1788576282.807023:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576282.807192:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576283.831903:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576283.836583:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788576284.855148:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576284.863364:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576285.879064:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576286.071158:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576286.071167:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576286.071172:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576286.071181:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8198 00000100:00080000:0.0:1788576286.071191:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576286.071198:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576286.071232:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616565504/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576286.071267:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576286.071271:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576286.071273:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576286.071277:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8198 00000100:00080000:0.0:1788576286.071280:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576286.071283:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576286.071301:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576286.071312:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d180 x1875449616565632/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576286.071321:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576286.071324:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576286.071326:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576286.071329:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8198 00000100:00080000:0.0:1788576286.071332:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576286.071335:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576286.071339:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576286.071346:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616565760/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788576286.071526:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616565504:192.168.202.113@tcp:250:kworker.0 00000800:00000100:2.0:1788576286.071582:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576286.071612:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576286/real 1788576286] req@ffff9ada4557e680 x1875449616565504/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576311 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576286.071627:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616565504/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576311 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576286.071636:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576286.071639:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576286.071643:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616565504:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788576286.071649:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616565632:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576286.071667:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576286.071693:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576286/real 1788576286] req@ffff9ada4557d180 x1875449616565632/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576311 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576286.071705:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d180 x1875449616565632/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576311 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576286.071712:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576286.071715:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576286.071718:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616565632:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576286.071723:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616565760:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576286.071740:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576286.071749:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576286/real 1788576286] req@ffff9ada4557ea00 x1875449616565760/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576311 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576286.071760:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616565760/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576311 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576286.071768:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576286.071770:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576286.071773:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616565760:192.168.202.113@tcp:38:kworker.0 00010000:00010000:1.0:1788576286.903774:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576287.612304:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576287.927074:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576288.631320:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576288.958188:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576289.657427:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576289.975071:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576290.679321:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576291.001712:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788576291.191207:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576291.191217:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788576291.191222:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576291.191233:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8203 00000100:00080000:1.0:1788576291.191243:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576291.191251:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788576291.191290:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c380 x1875449616565888/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788576291.191317:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576291.191320:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788576291.191322:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576291.191326:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8203 00000100:00080000:1.0:1788576291.191329:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576291.191333:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576291.191355:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576291.191367:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c700 x1875449616566016/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788576291.191377:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576291.191380:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788576291.191382:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576291.191386:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8203 00000100:00080000:1.0:1788576291.191389:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576291.191392:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576291.191397:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576291.191406:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f100 x1875449616566144/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788576291.192698:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616565888:192.168.202.113@tcp:250:kworker.0 00000800:00000100:2.0:1788576291.192757:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576291.192790:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576291/real 1788576291] req@ffff9ada4566c380 x1875449616565888/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576316 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576291.192809:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c380 x1875449616565888/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576316 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576291.192821:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576291.192825:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576291.192829:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616565888:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788576291.192836:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616566016:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576291.192877:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576291.192889:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576291/real 1788576291] req@ffff9ada4566c700 x1875449616566016/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576316 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576291.192900:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c700 x1875449616566016/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576316 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576291.192909:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576291.192911:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576291.192914:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616566016:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576291.192920:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616566144:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576291.192936:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576291.192944:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576291/real 1788576291] req@ffff9ada4566f100 x1875449616566144/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576316 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576291.192955:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f100 x1875449616566144/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576316 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576291.192968:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576291.192971:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576291.192975:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616566144:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788576291.705787:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576292.023167:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576292.728368:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576293.055772:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576293.750980:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576294.071566:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576294.776211:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576295.095397:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576295.798985:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576296.124836:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788576296.310987:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576296.311000:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788576296.311005:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576296.311016:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8208 00000100:00080000:1.0:1788576296.311027:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576296.311036:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788576296.311079:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d180 x1875449616566272/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788576296.311112:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576296.311116:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788576296.311118:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576296.311122:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8208 00000100:00080000:1.0:1788576296.311125:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576296.311130:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576296.311140:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576296.311151:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ca80 x1875449616566400/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788576296.311161:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576296.311164:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788576296.311165:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576296.311168:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8208 00000100:00080000:1.0:1788576296.311171:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576296.311175:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576296.311179:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576296.311187:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d500 x1875449616566528/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788576296.312351:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616566272:192.168.202.113@tcp:250:kworker.0 00000800:00000100:2.0:1788576296.312418:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576296.312453:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576296/real 1788576296] req@ffff9ada4566d180 x1875449616566272/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576321 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576296.312472:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d180 x1875449616566272/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576321 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576296.312493:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576296.312497:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576296.312501:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616566272:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788576296.312508:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616566400:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576296.312540:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576296.312552:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576296/real 1788576296] req@ffff9ada4566ca80 x1875449616566400/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576321 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576296.312565:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ca80 x1875449616566400/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576321 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576296.312572:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576296.312575:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576296.312579:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616566400:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576296.312586:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616566528:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576296.312603:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576296.312613:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576296/real 1788576296] req@ffff9ada4566d500 x1875449616566528/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576321 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576296.312624:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d500 x1875449616566528/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576321 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576296.312632:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576296.312636:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576296.312639:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616566528:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788576296.823356:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576297.143535:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576297.847423:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576298.171145:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576298.871538:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576299.191050:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576299.894991:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576300.219646:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576300.919477:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576301.273549:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576301.431709:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576301.431719:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576301.431723:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576301.431733:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8213 00000100:00080000:0.0:1788576301.431743:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576301.431751:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576301.431788:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557df80 x1875449616566656/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576301.431821:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576301.431824:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576301.431826:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576301.431829:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8213 00000100:00080000:0.0:1788576301.431833:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576301.431837:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576301.431845:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576301.431880:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d500 x1875449616566784/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576301.431890:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576301.431893:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576301.431894:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576301.431897:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8213 00000100:00080000:0.0:1788576301.431900:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576301.431904:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576301.431908:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576301.431920:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616566912/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788576301.432057:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616566656:192.168.202.113@tcp:250:kworker.0 00000800:00000100:2.0:1788576301.432114:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576301.432146:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576301/real 1788576301] req@ffff9ada4557df80 x1875449616566656/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576326 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576301.432162:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557df80 x1875449616566656/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576326 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576301.432171:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576301.432174:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576301.432177:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616566656:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788576301.432185:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616566784:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576301.432206:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576301.432217:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576301/real 1788576301] req@ffff9ada4557d500 x1875449616566784/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576326 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576301.432227:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d500 x1875449616566784/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576326 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576301.432234:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576301.432237:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576301.432240:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616566784:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576301.432245:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616566912:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576301.432262:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576301.432271:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576301/real 1788576301] req@ffff9ada4557ea00 x1875449616566912/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576326 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576301.432280:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616566912/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576326 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576301.432287:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576301.432290:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576301.432292:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616566912:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576301.943164:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576302.263097:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576302.968230:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576303.288562:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576303.994761:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576304.311391:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576305.014989:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576305.337026:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576306.039021:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576306.359199:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576306.552960:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576306.552974:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576306.552979:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576306.552992:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8219 00000100:00080000:0.0:1788576306.553004:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576306.553013:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576306.553065:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d180 x1875449616567040/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576306.553101:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576306.553105:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576306.553106:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576306.553110:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8219 00000100:00080000:0.0:1788576306.553114:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576306.553117:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576306.553130:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576306.553141:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616567168/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576306.553151:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576306.553154:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576306.553156:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576306.553160:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8219 00000100:00080000:0.0:1788576306.553174:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576306.553178:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576306.553183:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576306.553194:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616567296/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788576306.553393:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616567040:192.168.202.113@tcp:250:kworker.0 00000800:00000100:2.0:1788576306.553475:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576306.553518:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576306/real 1788576306] req@ffff9ada4557d180 x1875449616567040/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576331 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576306.553538:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d180 x1875449616567040/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576331 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576306.553555:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576306.553559:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576306.553563:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616567040:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788576306.553570:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616567168:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576306.553594:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576306.553607:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576306/real 1788576306] req@ffff9ada4557e680 x1875449616567168/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576331 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576306.553618:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616567168/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576331 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576306.553625:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576306.553629:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576306.553632:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616567168:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576306.553638:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616567296:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576306.553659:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576306.553669:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576306/real 1788576306] req@ffff9ada4557e300 x1875449616567296/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576331 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576306.553680:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616567296/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576331 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576306.553687:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576306.553690:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576306.553693:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616567296:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788576307.063021:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576307.383455:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576308.087730:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576308.407188:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576309.111466:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576309.431149:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576310.135354:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576310.455134:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576311.159931:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576311.479902:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788576311.672145:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576311.672160:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788576311.672171:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576311.672185:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8224 00000100:00080000:2.0:1788576311.672200:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576311.672210:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788576311.672263:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627100 x1875449616567424/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576311.672300:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576311.672304:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788576311.672305:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576311.672310:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8224 00000100:00080000:2.0:1788576311.672314:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576311.672318:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576311.672329:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576311.672343:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624700 x1875449616567552/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576311.672353:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576311.672356:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788576311.672358:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576311.672362:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8224 00000100:00080000:2.0:1788576311.672365:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576311.672368:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576311.672373:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576311.672384:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616567680/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576311.674465:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616567424:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576311.674708:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616567552:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576311.674742:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616567680:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788576312.183540:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576312.503109:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576313.206988:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576313.527801:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576314.231055:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576314.556214:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1788576314.743295:0:2370:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:2.0:1788576314.743339:0:2370:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1788576314.743354:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:3.0:1788576314.748371:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576311/real 1788576314] req@ffff9ada60627100 x1875449616567424/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576336 ref 2 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576314.748406:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627100 x1875449616567424/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576336 ref 2 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576314.748429:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576314.748435:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576314.748439:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616567424:192.168.202.113@tcp:250:kworker.0 00000800:00000100:2.0:1788576314.748780:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:2.0:1788576314.748806:0:2370:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:3.0:1788576314.748932:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576311/real 1788576314] req@ffff9ada60624700 x1875449616567552/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576336 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576314.748948:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624700 x1875449616567552/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576336 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576314.748956:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576314.748959:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576314.748962:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616567552:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576314.748969:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576311/real 1788576314] req@ffff9ada60626680 x1875449616567680/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576336 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576314.748981:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616567680/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576336 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576314.748988:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576314.748991:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576314.748994:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616567680:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788576315.258502:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576315.575106:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576316.279310:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576316.604573:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788576316.793542:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576316.793553:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788576316.793557:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576316.793567:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8229 00000100:00080000:2.0:1788576316.793578:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576316.793586:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788576316.793627:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627100 x1875449616567808/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576316.793657:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576316.793661:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788576316.793663:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576316.793666:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8229 00000100:00080000:2.0:1788576316.793670:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576316.793674:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576316.793684:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576316.793696:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616567936/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576316.793706:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576316.793708:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788576316.793710:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576316.793713:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8229 00000100:00080000:2.0:1788576316.793716:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576316.793719:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576316.793724:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576316.793733:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616568064/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576316.794719:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616567808:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576316.794965:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576316.795263:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576316/real 1788576316] req@ffff9ada60627100 x1875449616567808/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576341 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576316.795370:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627100 x1875449616567808/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576341 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576316.795444:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576316.795449:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576316.795453:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616567808:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576316.795460:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616567936:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576316.795493:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576316.795506:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576316/real 1788576316] req@ffff9ada60626a00 x1875449616567936/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576341 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576316.795518:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616567936/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576341 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576316.795527:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576316.795529:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576316.795532:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616567936:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576316.795538:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616568064:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576316.795558:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576316.795569:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576316/real 1788576316] req@ffff9ada60625500 x1875449616568064/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576341 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576316.795579:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616568064/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576341 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576316.795593:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576316.795596:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576316.795599:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616568064:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788576317.303250:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576317.623520:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576318.327011:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576318.649267:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576319.350982:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576319.672702:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576320.376975:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576320.695739:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576321.403025:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576321.722712:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576321.911266:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576321.911280:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576321.911285:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576321.911299:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8234 00000100:00080000:0.0:1788576321.911314:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576321.911324:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576321.911378:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616568192/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576321.911414:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576321.911418:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576321.911420:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576321.911424:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8234 00000100:00080000:0.0:1788576321.911428:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576321.911433:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576321.911445:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576321.911459:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616568320/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576321.911469:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576321.911473:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576321.911474:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576321.911478:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8234 00000100:00080000:0.0:1788576321.911481:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576321.911485:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576321.911490:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576321.911501:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616568448/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576321.911838:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616568192:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576321.911959:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576321.912005:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576321/real 1788576321] req@ffff9ada4557ed80 x1875449616568192/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576346 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576321.912026:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616568192/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576346 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576321.912037:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576321.912040:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576321.912044:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616568192:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576321.912051:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616568320:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576321.912074:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576321.912086:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576321/real 1788576321] req@ffff9ada4557fb80 x1875449616568320/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576346 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576321.912098:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616568320/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576346 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576321.912106:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576321.912110:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576321.912113:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616568320:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576321.912119:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616568448:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576321.912136:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576321.912146:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576321/real 1788576321] req@ffff9ada4557f800 x1875449616568448/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576346 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576321.912156:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616568448/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576346 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576321.912164:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576321.912167:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576321.912170:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616568448:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788576322.423446:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576322.746328:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576323.447328:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576323.767214:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576324.473036:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576324.795356:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576325.495024:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576325.815401:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576326.519177:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576326.848562:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576327.033424:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576327.033438:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576327.033443:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576327.033457:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8239 00000100:00080000:0.0:1788576327.033472:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576327.033481:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576327.033545:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616568576/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576327.033583:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576327.033587:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576327.033589:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576327.033593:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8239 00000100:00080000:0.0:1788576327.033597:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576327.033601:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576327.033612:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576327.033625:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616568704/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576327.033635:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576327.033639:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576327.033641:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576327.033645:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8239 00000100:00080000:0.0:1788576327.033649:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576327.033653:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576327.033658:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576327.033669:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616568832/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576327.033715:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616568576:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576327.033791:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576327.033830:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576327/real 1788576327] req@ffff9ada4557dc00 x1875449616568576/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576352 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576327.033850:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616568576/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576352 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576327.033909:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576327.033914:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576327.033918:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616568576:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576327.033926:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616568704:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576327.033951:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576327.033963:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576327/real 1788576327] req@ffff9ada4557d880 x1875449616568704/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576352 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576327.033975:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616568704/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576352 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576327.033983:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576327.033986:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576327.033989:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616568704:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576327.033994:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616568832:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576327.034012:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576327.034039:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576327/real 1788576327] req@ffff9ada4557ca80 x1875449616568832/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576352 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576327.034051:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616568832/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576352 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576327.034070:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576327.034073:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576327.034076:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616568832:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788576327.542979:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576327.863075:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576328.567655:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576328.887461:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576329.591402:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576329.911198:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576330.615024:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576330.944683:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576331.639050:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576331.960378:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788576332.153693:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788576332.153706:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788576332.153710:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788576332.153722:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8244 00000100:00080000:3.0:1788576332.153734:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788576332.153741:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788576332.153782:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3b80 x1875449616568960/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788576332.153817:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788576332.153821:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788576332.153823:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788576332.153826:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8244 00000100:00080000:3.0:1788576332.153830:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788576332.153834:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788576332.153844:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788576332.153882:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1c00 x1875449616569088/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788576332.153893:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788576332.153897:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788576332.153899:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788576332.153903:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8244 00000100:00080000:3.0:1788576332.153907:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788576332.153910:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788576332.153915:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788576332.153923:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0000 x1875449616569216/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788576332.155055:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616568960:192.168.202.113@tcp:250:kworker.0 00000800:00000100:2.0:1788576332.155152:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576332.155189:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576332/real 1788576332] req@ffff9ada604e3b80 x1875449616568960/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576357 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576332.155205:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3b80 x1875449616568960/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576357 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576332.155215:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576332.155218:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576332.155221:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616568960:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788576332.155228:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616569088:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576332.155249:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576332.155260:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576332/real 1788576332] req@ffff9ada604e1c00 x1875449616569088/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576357 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576332.155271:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1c00 x1875449616569088/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576357 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576332.155279:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576332.155282:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576332.155285:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616569088:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576332.155291:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616569216:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576332.155312:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576332.155322:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576332/real 1788576332] req@ffff9ada604e0000 x1875449616569216/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576357 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576332.155333:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0000 x1875449616569216/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576357 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576332.155339:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576332.155342:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576332.155345:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616569216:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576332.663328:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576332.983210:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576333.689820:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576334.007158:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576334.711670:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576335.034393:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576335.735017:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576336.055399:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576336.761431:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576337.079082:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788576337.271092:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576337.271102:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788576337.271105:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576337.271115:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8249 00000100:00080000:2.0:1788576337.271126:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576337.271132:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788576337.271181:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616569344/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576337.271211:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576337.271214:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788576337.271216:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576337.271219:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8249 00000100:00080000:2.0:1788576337.271223:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576337.271227:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576337.271236:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576337.271247:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616569472/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576337.271255:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576337.271258:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788576337.271260:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576337.271263:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8249 00000100:00080000:2.0:1788576337.271265:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576337.271268:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576337.271272:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576337.271281:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616569600/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576337.271362:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616569344:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576337.271434:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576337.271468:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576337/real 1788576337] req@ffff9ada60624a80 x1875449616569344/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576362 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576337.271486:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616569344/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576362 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576337.271496:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576337.271501:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576337.271517:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616569344:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576337.271526:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616569472:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576337.271548:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576337.271560:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576337/real 1788576337] req@ffff9ada60625880 x1875449616569472/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576362 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576337.271570:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616569472/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576362 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576337.271577:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576337.271580:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576337.271583:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616569472:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576337.271589:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616569600:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576337.271606:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576337.271616:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576337/real 1788576337] req@ffff9ada60624380 x1875449616569600/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576362 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576337.271626:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616569600/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576362 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576337.271633:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576337.271636:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576337.271639:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616569600:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788576337.783228:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576338.126746:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576338.807043:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576339.128291:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576339.831450:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576340.162444:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576340.855529:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576341.175153:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576341.880912:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576342.200913:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788576342.391241:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576342.391254:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788576342.391260:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576342.391272:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8254 00000100:00080000:2.0:1788576342.391288:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576342.391297:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788576342.391352:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616569728/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576342.391391:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576342.391395:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788576342.391397:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576342.391401:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8254 00000100:00080000:2.0:1788576342.391405:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576342.391409:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576342.391423:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576342.391436:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616569856/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576342.391446:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576342.391450:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788576342.391451:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576342.391455:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8254 00000100:00080000:2.0:1788576342.391459:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576342.391463:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576342.391468:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576342.391478:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616569984/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576342.391581:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616569728:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576342.391663:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576342.391706:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576342/real 1788576342] req@ffff9ada60625c00 x1875449616569728/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576367 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576342.391725:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616569728/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576367 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576342.391764:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576342.391768:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576342.391772:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616569728:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576342.391780:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616569856:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576342.391808:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576342.391819:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576342/real 1788576342] req@ffff9ada60624e00 x1875449616569856/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576367 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576342.391831:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616569856/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576367 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576342.391839:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576342.391842:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576342.391845:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616569856:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576342.391878:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616569984:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576342.391912:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576342.391922:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576342/real 1788576342] req@ffff9ada60627480 x1875449616569984/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576367 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576342.391933:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616569984/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576367 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576342.391941:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576342.391945:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576342.391948:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616569984:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788576342.903019:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576343.223240:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576343.928181:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576344.248201:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576344.954012:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576345.273381:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576345.975250:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576346.296153:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576347.002840:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576347.322609:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788576347.511092:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576347.511108:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788576347.511114:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576347.511127:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8260 00000100:00080000:1.0:1788576347.511142:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576347.511152:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788576347.511213:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ea00 x1875449616570112/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576347.511308:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616570112:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576347.511394:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00080000:1.0:1788576347.511429:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576347.511434:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788576347.511436:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576347.511441:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8260 00000100:00080000:1.0:1788576347.511445:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576347.511449:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576347.511461:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576347.511475:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f480 x1875449616570240/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576347.511902:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576347/real 1788576347] req@ffff9ada4566ea00 x1875449616570112/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576372 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576347.511935:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ea00 x1875449616570112/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576372 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576347.511946:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576347.511950:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576347.515074:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616570112:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576347.515127:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616570240:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576347.515218:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576347.515254:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576347/real 1788576347] req@ffff9ada4566f480 x1875449616570240/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576372 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576347.515276:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f480 x1875449616570240/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576372 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576347.515288:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576347.515292:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576347.515295:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616570240:192.168.202.113@tcp:38:kworker.0 00000100:00080000:1.0:1788576347.516105:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576347.516112:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788576347.516115:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576347.516124:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8260 00000100:00080000:1.0:1788576347.516133:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576347.516141:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576347.516156:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576347.517232:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e300 x1875449616570368/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576347.517378:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616570368:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576347.517444:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576347.517474:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576347/real 1788576347] req@ffff9ada4566e300 x1875449616570368/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576372 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576347.517498:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e300 x1875449616570368/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576372 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576347.517507:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576347.517511:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576347.517514:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616570368:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788576348.022989:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576348.343623:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576349.047377:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576349.369881:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576350.070951:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576350.391126:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576351.095045:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576351.423204:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576352.119494:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576352.441098:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788576352.635279:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576352.635295:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788576352.635301:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576352.635434:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8265 00000100:00080000:2.0:1788576352.635450:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576352.635462:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788576352.635517:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616570496/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576352.635552:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576352.635557:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788576352.635559:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576352.635564:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8265 00000100:00080000:2.0:1788576352.635567:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576352.635572:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576352.635584:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576352.635600:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616570624/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576352.635608:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576352.635612:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788576352.635614:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576352.635617:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8265 00000100:00080000:2.0:1788576352.635621:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576352.635625:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576352.635630:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576352.635640:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616570752/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576352.635754:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616570496:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576352.635886:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576352.636206:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576352/real 1788576352] req@ffff9ada60625f80 x1875449616570496/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576377 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576352.636231:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616570496/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576377 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576352.636243:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576352.636247:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576352.636251:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616570496:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576352.636260:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616570624:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576352.636290:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576352.636305:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576352/real 1788576352] req@ffff9ada60625180 x1875449616570624/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576377 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576352.636326:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616570624/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576377 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576352.636497:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576352.636501:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576352.636505:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616570624:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576352.636512:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616570752:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576352.636537:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576352.636549:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576352/real 1788576352] req@ffff9ada60626d80 x1875449616570752/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576377 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576352.636559:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616570752/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576377 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576352.636567:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576352.636571:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576352.636574:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616570752:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788576353.143457:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576353.464402:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576354.167058:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576354.487572:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576355.192082:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576355.512708:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576356.216345:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576356.535564:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576357.240272:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576357.561010:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788576357.751080:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788576357.751097:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788576357.751102:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788576357.751129:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8270 00000100:00080000:3.0:1788576357.751144:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788576357.751154:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788576357.751217:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2300 x1875449616570880/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788576357.751273:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788576357.751278:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788576357.751280:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788576357.751284:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8270 00000100:00080000:3.0:1788576357.751288:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788576357.751293:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788576357.751304:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788576357.751317:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0a80 x1875449616571008/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788576357.751327:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788576357.751330:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788576357.751332:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788576357.751336:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8270 00000100:00080000:3.0:1788576357.751339:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788576357.751343:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788576357.751348:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788576357.751358:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2d80 x1875449616571136/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788576357.751526:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616570880:192.168.202.113@tcp:250:kworker.0 00000800:00000100:2.0:1788576357.751610:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576357.751655:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576357/real 1788576357] req@ffff9ada604e2300 x1875449616570880/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576382 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576357.751674:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2300 x1875449616570880/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576382 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576357.751685:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576357.751689:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576357.751693:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616570880:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788576357.751701:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616571008:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576357.751739:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576357.751751:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576357/real 1788576357] req@ffff9ada604e0a80 x1875449616571008/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576382 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576357.751763:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0a80 x1875449616571008/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576382 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576357.751770:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576357.751774:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576357.751777:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616571008:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576357.751783:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616571136:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576357.751806:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576357.751817:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576357/real 1788576357] req@ffff9ada604e2d80 x1875449616571136/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576382 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576357.751829:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2d80 x1875449616571136/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576382 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576357.751836:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576357.751839:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576357.751842:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616571136:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576358.263048:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576358.583275:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576359.287567:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576359.607784:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576360.311501:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576360.631264:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576361.335068:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576361.656495:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576362.365070:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576362.681161:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788576362.876103:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576362.876119:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788576362.876125:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576362.876149:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8275 00000100:00080000:1.0:1788576362.876167:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576362.876179:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788576362.876239:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f800 x1875449616571264/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788576362.876281:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576362.876285:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788576362.876287:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576362.876292:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8275 00000100:00080000:1.0:1788576362.876296:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576362.876300:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576362.876312:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576362.876328:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566dc00 x1875449616571392/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788576362.876344:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576362.876347:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788576362.876349:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576362.876353:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8275 00000100:00080000:1.0:1788576362.876356:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576362.876360:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576362.876365:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576362.876376:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ed80 x1875449616571520/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788576362.877018:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616571264:192.168.202.113@tcp:250:kworker.0 00000800:00000100:2.0:1788576362.877108:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576362.877169:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576362/real 1788576362] req@ffff9ada4566f800 x1875449616571264/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576387 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576362.877189:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f800 x1875449616571264/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576387 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576362.877201:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576362.877205:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576362.877209:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616571264:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788576362.877217:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616571392:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576362.877246:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576362.877260:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576362/real 1788576362] req@ffff9ada4566dc00 x1875449616571392/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576387 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576362.877272:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566dc00 x1875449616571392/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576387 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576362.877280:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576362.877284:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576362.877287:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616571392:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576362.877294:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616571520:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576362.877315:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576362.877326:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576362/real 1788576362] req@ffff9ada4566ed80 x1875449616571520/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576387 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576362.877336:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ed80 x1875449616571520/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576387 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576362.877345:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576362.877348:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576362.877352:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616571520:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788576363.383308:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576363.705040:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576364.409157:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576364.727224:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576365.431267:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576365.754634:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576366.455096:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576366.777294:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576367.480250:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576367.799608:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576367.991019:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576367.991034:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576367.991040:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576367.991053:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8280 00000100:00080000:0.0:1788576367.991080:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576367.991091:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576367.991142:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616571648/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576367.991178:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576367.991182:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576367.991184:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576367.991188:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8280 00000100:00080000:0.0:1788576367.991192:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576367.991196:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576367.991207:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576367.991221:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ce00 x1875449616571776/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576367.991231:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576367.991235:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576367.991236:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576367.991240:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8280 00000100:00080000:0.0:1788576367.991243:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576367.991246:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576367.991251:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576367.991261:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c000 x1875449616571904/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788576367.994364:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616571648:192.168.202.113@tcp:250:kworker.0 00000800:00000100:2.0:1788576367.994472:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576367.994528:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576367/real 1788576367] req@ffff9ada4557c700 x1875449616571648/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576392 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576367.994552:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616571648/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576392 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576367.994565:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576367.994570:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576367.994574:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616571648:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788576367.994582:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616571776:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576367.994621:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576367.994635:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576367/real 1788576367] req@ffff9ada4557ce00 x1875449616571776/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576392 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576367.994648:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ce00 x1875449616571776/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576392 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576367.994655:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576367.994659:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576367.994662:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616571776:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576367.994670:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616571904:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576367.994692:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576367.994702:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576367/real 1788576367] req@ffff9ada4557c000 x1875449616571904/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576392 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576367.994714:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c000 x1875449616571904/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576392 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576367.994727:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576367.994730:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576367.994733:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616571904:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788576368.503084:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576368.829183:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576369.527025:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576369.847185:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576370.551360:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576370.871142:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576371.575319:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576371.895156:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576372.599603:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576372.919788:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788576373.117890:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788576373.117905:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788576373.117911:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788576373.117924:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8285 00000100:00080000:3.0:1788576373.117939:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788576373.117948:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788576373.118001:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1880 x1875449616572032/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788576373.118035:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788576373.118040:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788576373.118042:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788576373.118046:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8285 00000100:00080000:3.0:1788576373.118050:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788576373.118053:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788576373.118065:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788576373.118081:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3800 x1875449616572160/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788576373.118091:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788576373.118094:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788576373.118096:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788576373.118100:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8285 00000100:00080000:3.0:1788576373.118103:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788576373.118107:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788576373.118112:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788576373.118124:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1180 x1875449616572288/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788576373.118232:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616572032:192.168.202.113@tcp:250:kworker.0 00000800:00000100:2.0:1788576373.118314:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576373.118357:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576373/real 1788576373] req@ffff9ada604e1880 x1875449616572032/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576398 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576373.118376:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1880 x1875449616572032/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576398 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576373.118388:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576373.118392:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576373.118395:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616572032:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788576373.118403:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616572160:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576373.118429:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576373.118442:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576373/real 1788576373] req@ffff9ada604e3800 x1875449616572160/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576398 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576373.118452:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3800 x1875449616572160/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576398 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576373.118461:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576373.118464:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576373.118467:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616572160:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576373.118473:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616572288:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576373.118492:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576373.118502:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576373/real 1788576373] req@ffff9ada604e1180 x1875449616572288/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576398 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576373.118512:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1180 x1875449616572288/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576398 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576373.118521:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576373.118524:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576373.118527:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616572288:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788576373.622995:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576373.943307:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576374.648252:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576374.967261:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576375.671060:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576375.993456:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576376.695162:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576377.021597:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576377.719392:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576378.039310:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788576378.231762:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576378.231777:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788576378.231783:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576378.231796:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8290 00000100:00080000:1.0:1788576378.231810:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576378.231820:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788576378.231895:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c000 x1875449616572416/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788576378.231930:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576378.231935:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788576378.231937:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576378.231941:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8290 00000100:00080000:1.0:1788576378.231945:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576378.231949:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576378.231961:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576378.231975:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566df80 x1875449616572544/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788576378.231985:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576378.231989:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788576378.231991:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576378.232006:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8290 00000100:00080000:1.0:1788576378.232009:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576378.232013:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576378.232019:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576378.232030:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566fb80 x1875449616572672/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788576378.232135:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616572416:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788576378.232252:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616572544:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576378.232294:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616572672:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576378.747331:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576379.065900:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576379.766990:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576380.088778:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576380.791199:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576381.111170:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1788576381.304074:0:2372:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:1.0:1788576381.304140:0:2372:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1788576381.304163:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:1.0:1788576381.304211:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:1.0:1788576381.304223:0:2372:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:2.0:1788576381.304709:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576378/real 1788576381] req@ffff9ada4566c000 x1875449616572416/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576403 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576381.304760:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c000 x1875449616572416/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576403 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576381.304771:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576381.304776:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576381.304780:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616572416:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788576381.304788:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576378/real 1788576381] req@ffff9ada4566df80 x1875449616572544/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576403 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576381.304801:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566df80 x1875449616572544/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576403 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576381.304808:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576381.304811:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576381.304814:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616572544:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576381.304820:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576378/real 1788576381] req@ffff9ada4566fb80 x1875449616572672/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576403 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576381.304830:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566fb80 x1875449616572672/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576403 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576381.304837:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576381.305305:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576381.305312:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616572672:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788576381.815001:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576382.138114:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576382.840201:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576383.164572:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788576383.353328:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576383.353339:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788576383.353342:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576383.353354:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8295 00000100:00080000:2.0:1788576383.353379:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576383.353395:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788576383.353429:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616572800/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576383.353468:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576383.353473:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788576383.353475:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576383.353479:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8295 00000100:00080000:2.0:1788576383.353483:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576383.353487:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576383.353496:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576383.353510:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627800 x1875449616572928/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576383.353520:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576383.353523:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788576383.353525:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576383.353529:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8295 00000100:00080000:2.0:1788576383.353533:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576383.353537:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576383.353542:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576383.353552:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626300 x1875449616573056/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576383.353689:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616572800:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576383.353789:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576383.354689:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576383/real 1788576383] req@ffff9ada60627b80 x1875449616572800/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576408 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576383.354725:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616572800/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576408 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576383.354737:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576383.354741:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576383.354745:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616572800:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576383.354753:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616572928:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576383.354785:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576383.355085:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576383/real 1788576383] req@ffff9ada60627800 x1875449616572928/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576408 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576383.355098:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627800 x1875449616572928/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576408 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576383.355106:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576383.355110:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576383.355113:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616572928:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576383.355120:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616573056:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576383.355157:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576383.355169:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576383/real 1788576383] req@ffff9ada60626300 x1875449616573056/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576408 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576383.355180:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626300 x1875449616573056/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576408 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576383.355186:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576383.355189:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576383.355192:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616573056:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788576383.863010:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576384.183283:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576384.887196:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576385.207090:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576385.911286:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576386.231532:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576386.935300:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576387.255606:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576387.960977:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576388.279078:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788576388.471288:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576388.471298:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788576388.471303:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576388.471314:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8301 00000100:00080000:1.0:1788576388.471323:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576388.471332:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788576388.471374:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566fb80 x1875449616573184/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788576388.471409:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576388.471412:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788576388.471414:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576388.471418:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8301 00000100:00080000:1.0:1788576388.471421:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576388.471424:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576388.471433:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576388.471452:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566df80 x1875449616573312/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788576388.471471:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576388.471474:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788576388.471476:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576388.471480:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8301 00000100:00080000:1.0:1788576388.471483:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576388.471487:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576388.471491:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576388.471502:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c000 x1875449616573440/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576388.471650:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616573184:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576388.471742:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576388.471787:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576388/real 1788576388] req@ffff9ada4566fb80 x1875449616573184/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576413 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576388.471805:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566fb80 x1875449616573184/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576413 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576388.471816:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576388.471820:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576388.471825:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616573184:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576388.471832:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616573312:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576388.471884:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576388.471898:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576388/real 1788576388] req@ffff9ada4566df80 x1875449616573312/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576413 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576388.471910:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566df80 x1875449616573312/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576413 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576388.471918:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576388.471922:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576388.471924:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616573312:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576388.471930:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616573440:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576388.471950:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576388.471960:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576388/real 1788576388] req@ffff9ada4566c000 x1875449616573440/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576413 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576388.471972:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c000 x1875449616573440/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576413 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576388.471980:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576388.471983:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576388.471986:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616573440:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788576388.983006:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576389.304824:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576390.007098:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576390.328685:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576391.031361:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576391.351592:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576392.055000:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576392.375444:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576393.079092:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576393.399100:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788576393.591790:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788576393.591800:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788576393.591804:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788576393.591814:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8306 00000100:00080000:3.0:1788576393.591823:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788576393.591830:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788576393.591886:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0700 x1875449616573568/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788576393.591914:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788576393.591917:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788576393.591919:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788576393.591922:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8306 00000100:00080000:3.0:1788576393.591926:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788576393.591929:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788576393.591938:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788576393.591950:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3480 x1875449616573696/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788576393.591959:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788576393.591962:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788576393.591963:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788576393.591967:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8306 00000100:00080000:3.0:1788576393.591970:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788576393.591974:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788576393.591978:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788576393.591988:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0380 x1875449616573824/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788576393.592415:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616573568:192.168.202.113@tcp:250:kworker.0 00000800:00000100:2.0:1788576393.592481:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576393.592516:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576393/real 1788576393] req@ffff9ada604e0700 x1875449616573568/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576418 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576393.592533:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0700 x1875449616573568/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576418 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576393.592549:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576393.592553:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576393.592557:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616573568:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788576393.592564:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616573696:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576393.592585:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576393.592645:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576393/real 1788576393] req@ffff9ada604e3480 x1875449616573696/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576418 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576393.592657:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3480 x1875449616573696/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576418 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576393.592665:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576393.592668:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576393.592671:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616573696:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576393.592678:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616573824:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576393.592698:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576393.592708:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576393/real 1788576393] req@ffff9ada604e0380 x1875449616573824/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576418 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576393.592718:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0380 x1875449616573824/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576418 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576393.592724:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576393.592727:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576393.592730:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616573824:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576394.103655:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576394.423121:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576395.127081:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576395.447643:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576396.151084:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576396.471121:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576397.176616:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576397.504411:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576398.199115:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576398.520033:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788576398.711404:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576398.711416:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788576398.711420:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576398.711430:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8311 00000100:00080000:1.0:1788576398.711442:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576398.711450:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788576398.711490:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ed80 x1875449616573952/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788576398.711525:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576398.711529:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788576398.711531:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576398.711535:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8311 00000100:00080000:1.0:1788576398.711539:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576398.711543:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576398.711565:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576398.711579:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566dc00 x1875449616574080/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788576398.711589:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576398.711592:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788576398.711594:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576398.711598:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8311 00000100:00080000:1.0:1788576398.711602:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576398.711606:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576398.711610:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576398.711621:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f800 x1875449616574208/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788576398.711801:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616573952:192.168.202.113@tcp:250:kworker.0 00000800:00000100:2.0:1788576398.711906:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576398.711943:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576398/real 1788576398] req@ffff9ada4566ed80 x1875449616573952/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576423 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576398.711961:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ed80 x1875449616573952/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576423 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576398.711972:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576398.711976:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576398.711979:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616573952:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788576398.711986:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616574080:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576398.712008:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576398.712021:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576398/real 1788576398] req@ffff9ada4566dc00 x1875449616574080/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576423 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576398.712032:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566dc00 x1875449616574080/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576423 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576398.712040:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576398.712042:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576398.712045:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616574080:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576398.712051:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616574208:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576398.712069:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576398.712079:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576398/real 1788576398] req@ffff9ada4566f800 x1875449616574208/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576423 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576398.712090:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f800 x1875449616574208/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576423 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576398.712098:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576398.712101:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576398.712104:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616574208:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576399.223002:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576399.549949:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576400.249743:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576400.568156:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576401.271227:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576401.591269:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576402.295020:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576402.615114:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576403.319772:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576403.643060:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576403.831253:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576403.831264:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576403.831268:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576403.831279:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8316 00000100:00080000:0.0:1788576403.831291:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576403.831299:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576403.831344:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c380 x1875449616574336/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576403.831378:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576403.831382:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576403.831384:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576403.831387:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8316 00000100:00080000:0.0:1788576403.831391:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576403.831395:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576403.831404:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576403.831417:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616574464/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576403.831427:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576403.831430:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576403.831431:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576403.831435:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8316 00000100:00080000:0.0:1788576403.831438:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576403.831442:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576403.831447:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576403.831460:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c000 x1875449616574592/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1788576403.831604:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616574336:192.168.202.113@tcp:250:kworker.0 00000800:00000100:2.0:1788576403.831667:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576403.831700:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576403/real 1788576403] req@ffff9ada4557c380 x1875449616574336/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576428 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576403.831718:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c380 x1875449616574336/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576428 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576403.831728:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576403.831732:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576403.831736:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616574336:192.168.202.113@tcp:250:kworker.0 00000100:00100000:2.0:1788576403.831743:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616574464:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576403.831766:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576403.831778:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576403/real 1788576403] req@ffff9ada4557f480 x1875449616574464/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576428 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576403.831789:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616574464/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576428 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576403.831803:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576403.831806:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576403.831810:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616574464:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576403.831815:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616574592:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576403.831836:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:2.0:1788576403.831846:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576403/real 1788576403] req@ffff9ada4557c000 x1875449616574592/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576428 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788576403.832036:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c000 x1875449616574592/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576428 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1788576403.832044:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1788576403.832047:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:2.0:1788576403.832050:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616574592:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788576404.345082:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576404.663163:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576405.367048:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576405.688661:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576406.393174:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576406.713566:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576407.415002:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576407.735114:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576408.439023:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576408.759110:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788576408.951307:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576408.951317:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788576408.951321:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576408.951331:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8321 00000100:00080000:2.0:1788576408.951342:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576408.951350:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788576408.951388:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626300 x1875449616574720/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576408.951425:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576408.951429:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788576408.951431:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576408.951435:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8321 00000100:00080000:2.0:1788576408.951438:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576408.951442:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576408.951452:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576408.951476:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627800 x1875449616574848/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576408.951486:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576408.951489:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788576408.951491:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576408.951495:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8321 00000100:00080000:2.0:1788576408.951498:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576408.951502:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576408.951506:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576408.951516:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627b80 x1875449616574976/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576408.951618:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616574720:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576408.951689:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576408.951721:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576408/real 1788576408] req@ffff9ada60626300 x1875449616574720/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576433 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576408.951741:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626300 x1875449616574720/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576433 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576408.951751:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576408.951754:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576408.951758:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616574720:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576408.951764:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616574848:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576408.951786:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576408.951797:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576408/real 1788576408] req@ffff9ada60627800 x1875449616574848/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576433 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576408.951814:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627800 x1875449616574848/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576433 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576408.951821:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576408.951824:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576408.951827:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616574848:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576408.951832:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616574976:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576408.951851:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576408.951887:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576408/real 1788576408] req@ffff9ada60627b80 x1875449616574976/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576433 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576408.951899:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627b80 x1875449616574976/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576433 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576408.951907:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576408.951910:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576408.951913:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616574976:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788576409.462982:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576409.786238:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576410.487773:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576410.807590:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576411.511148:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576411.831623:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576412.535597:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576412.855339:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576413.559027:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576413.879095:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576414.071898:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576414.071909:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576414.071913:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576414.071925:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8326 00000100:00080000:0.0:1788576414.071934:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576414.071942:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576414.073624:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ce00 x1875449616575104/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576414.073661:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576414.073666:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576414.073669:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576414.073675:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8326 00000100:00080000:0.0:1788576414.073683:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576414.073691:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576414.073701:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576414.073714:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616575232/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576414.073723:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576414.073727:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576414.073729:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576414.073733:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8326 00000100:00080000:0.0:1788576414.073736:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576414.073740:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576414.073744:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576414.073753:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616575360/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576414.073815:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616575104:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576414.073903:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576414.073937:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576414/real 1788576414] req@ffff9ada4557ce00 x1875449616575104/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576439 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576414.073954:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ce00 x1875449616575104/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576439 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576414.073964:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576414.073968:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576414.073972:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616575104:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576414.073979:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616575232:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576414.074000:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576414.074013:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576414/real 1788576414] req@ffff9ada4557c700 x1875449616575232/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576439 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576414.074025:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616575232/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576439 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576414.074033:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576414.074036:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576414.074039:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616575232:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576414.074044:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616575360:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576414.074063:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576414.074085:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576414/real 1788576414] req@ffff9ada4557ca80 x1875449616575360/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576439 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576414.074103:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616575360/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576439 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576414.074110:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576414.074113:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576414.074116:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616575360:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576414.583207:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576414.904120:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576415.607720:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576415.928931:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576416.631067:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576416.952136:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576417.655030:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576417.976136:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576418.679083:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576419.003180:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576419.191689:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576419.191704:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576419.191710:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576419.191725:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8331 00000100:00080000:0.0:1788576419.191740:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576419.191750:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576419.191804:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616575488/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576419.191895:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616575488:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576419.191985:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00080000:0.0:1788576419.192059:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576419.192064:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576419.192066:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576419.192071:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8331 00000100:00080000:0.0:1788576419.192075:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576419.192080:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576419.192101:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576419.192116:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616575616/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576419.206946:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576419/real 1788576419] req@ffff9ada4557d880 x1875449616575488/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576444 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576419.206985:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616575488/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576444 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576419.207000:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576419.207007:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576419.207012:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616575488:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576419.207058:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616575616:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576419.207157:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576419.207179:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576419/real 1788576419] req@ffff9ada4557dc00 x1875449616575616/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576444 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576419.207192:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616575616/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576444 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576419.207201:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576419.207206:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576419.207209:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616575616:192.168.202.113@tcp:38:kworker.0 00000100:00080000:0.0:1788576419.207502:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576419.207508:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576419.207512:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576419.207519:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8331 00000100:00080000:0.0:1788576419.207527:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576419.207536:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576419.207585:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576419.207621:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616575744/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576419.208499:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616575744:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576419.208563:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576419.208587:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576419/real 1788576419] req@ffff9ada4557f800 x1875449616575744/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576444 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576419.208602:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616575744/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576444 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576419.208613:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576419.208617:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576419.208620:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616575744:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576419.704060:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576420.027210:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576420.727032:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576421.047219:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576421.754149:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576422.071621:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576422.774981:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576423.096632:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576423.800582:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576424.119203:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576424.311385:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576424.311401:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576424.311407:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576424.311421:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8336 00000100:00080000:0.0:1788576424.311435:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576424.311446:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576424.311500:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616575872/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576424.311538:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576424.311556:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576424.311559:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576424.311564:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8336 00000100:00080000:0.0:1788576424.311568:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576424.311571:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576424.311584:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576424.311597:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616576000/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576424.311608:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576424.311611:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576424.311613:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576424.311617:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8336 00000100:00080000:0.0:1788576424.311621:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576424.311626:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576424.311630:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576424.311641:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616576128/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576424.311743:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616575872:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576424.311824:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576424.311894:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576424/real 1788576424] req@ffff9ada4557fb80 x1875449616575872/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576449 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576424.311917:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616575872/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576449 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576424.311934:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576424.311940:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576424.311944:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616575872:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576424.311952:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616576000:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576424.311980:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576424.311994:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576424/real 1788576424] req@ffff9ada4557ed80 x1875449616576000/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576449 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576424.312006:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616576000/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576449 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576424.312014:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576424.312018:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576424.312021:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616576000:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576424.312027:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616576128:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576424.312046:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576424.312118:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576424/real 1788576424] req@ffff9ada4557e300 x1875449616576128/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576449 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576424.312134:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616576128/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576449 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576424.312143:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576424.312147:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576424.312150:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616576128:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576424.823812:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576425.143176:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576425.847085:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576426.167099:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576426.871339:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576427.191070:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576427.895346:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576428.215890:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576428.919606:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576429.240238:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576429.430994:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576429.431246:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576429.431251:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576429.431262:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8341 00000100:00080000:0.0:1788576429.431302:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576429.431310:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576429.431353:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616576256/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576429.431435:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616576256:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576429.431519:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00080000:0.0:1788576429.431612:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576429.431616:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576429.431618:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576429.431622:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8341 00000100:00080000:0.0:1788576429.431783:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576429.431790:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576429.431801:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576429.431816:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d180 x1875449616576384/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576429.432025:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576429/real 1788576429] req@ffff9ada4557e680 x1875449616576256/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576454 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576429.432044:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616576256/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576454 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576429.433829:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576429.433839:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576429.433846:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616576256:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576429.433920:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616576384:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576429.433991:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576429.434028:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576429/real 1788576429] req@ffff9ada4557d180 x1875449616576384/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576454 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576429.434046:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d180 x1875449616576384/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576454 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576429.434062:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576429.434065:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576429.434069:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616576384:192.168.202.113@tcp:38:kworker.0 00000100:00080000:0.0:1788576429.438982:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576429.438995:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576429.439000:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576429.439023:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8341 00000100:00080000:0.0:1788576429.439034:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576429.439042:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576429.439743:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576429.439783:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616576512/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576429.440364:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616576512:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576429.440450:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576429.440577:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576429/real 1788576429] req@ffff9ada4557ea00 x1875449616576512/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576454 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576429.440598:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616576512/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576454 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576429.440614:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576429.440618:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576429.440622:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616576512:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576429.943206:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576430.269229:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576430.967576:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576431.287526:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576431.991896:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576432.312139:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576433.018521:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576433.335188:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576434.039528:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576434.361308:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788576434.551258:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576434.551273:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788576434.551279:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576434.551292:0:92891:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8347 00000100:00080000:2.0:1788576434.551307:0:92891:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576434.551317:0:92891:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788576434.551378:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626d80 x1875449616576640/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576434.551415:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576434.551419:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788576434.551421:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576434.551426:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8347 00000100:00080000:2.0:1788576434.551430:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576434.551434:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576434.551447:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576434.551461:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616576768/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576434.551471:0:92891:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576434.551475:0:92891:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788576434.551478:0:92891:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576434.551483:0:92891:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8347 00000100:00080000:2.0:1788576434.551486:0:92891:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576434.551490:0:92891:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576434.551495:0:92891:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576434.551505:0:92891:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616576896/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576434.551640:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616576640:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576434.551745:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576434.551795:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576434/real 1788576434] req@ffff9ada60626d80 x1875449616576640/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576459 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576434.551818:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626d80 x1875449616576640/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576459 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576434.552409:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576434.552418:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576434.552423:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616576640:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576434.552434:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616576768:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576434.552469:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576434.552486:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576434/real 1788576434] req@ffff9ada60625180 x1875449616576768/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576459 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576434.552499:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616576768/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576459 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576434.552514:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576434.552517:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576434.552521:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616576768:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576434.552527:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616576896:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576434.552547:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576434.552557:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576434/real 1788576434] req@ffff9ada60625f80 x1875449616576896/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576459 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576434.552568:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616576896/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576459 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576434.552576:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576434.552579:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576434.552583:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616576896:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576435.064170:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576435.385972:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576436.087688:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576436.407110:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576437.111275:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576437.431184:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576438.135050:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576438.455842:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576439.160429:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576439.479210:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576439.672081:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576439.672094:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576439.672098:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576439.672109:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8352 00000100:00080000:0.0:1788576439.672122:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576439.672130:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576439.672193:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d500 x1875449616577024/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576439.672225:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576439.672229:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576439.672231:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576439.672235:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8352 00000100:00080000:0.0:1788576439.672238:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576439.672242:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576439.672251:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576439.672264:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557df80 x1875449616577152/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576439.672274:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576439.672277:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576439.672279:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576439.672282:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8352 00000100:00080000:0.0:1788576439.672286:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576439.672290:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576439.672295:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576439.672305:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616577280/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576439.672410:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616577024:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576439.672481:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576439.672519:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576439/real 1788576439] req@ffff9ada4557d500 x1875449616577024/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576464 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576439.672537:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d500 x1875449616577024/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576464 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576439.672547:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576439.672550:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576439.672554:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616577024:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576439.672560:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616577152:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576439.672594:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576439.672607:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576439/real 1788576439] req@ffff9ada4557df80 x1875449616577152/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576464 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576439.672618:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557df80 x1875449616577152/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576464 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576439.672626:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576439.672629:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576439.672631:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616577152:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576439.672637:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616577280:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576439.672654:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576439.672663:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576439/real 1788576439] req@ffff9ada4557ea00 x1875449616577280/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576464 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576439.672675:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616577280/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576464 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576439.672682:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576439.672686:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576439.672700:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616577280:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788576440.184405:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576440.503226:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576441.207885:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576441.527240:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576442.231283:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576442.551320:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576443.255207:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576443.575168:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576444.279029:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576444.603365:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576444.794581:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576444.794596:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576444.794611:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576444.794626:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8357 00000100:00080000:0.0:1788576444.794640:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576444.794651:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576444.794706:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d180 x1875449616577408/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576444.794746:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576444.794750:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576444.794752:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576444.794756:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8357 00000100:00080000:0.0:1788576444.794760:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576444.794765:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576444.794778:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576444.794793:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616577536/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576444.794807:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576444.794810:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576444.794813:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1788576444.794818:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616577408:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788576444.794819:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8357 00000100:00080000:0.0:1788576444.794825:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576444.794831:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576444.794836:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576444.794850:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616577664/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576444.794984:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616577536:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576444.795020:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616577664:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788576445.302966:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576445.625814:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576446.327360:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576446.651613:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576447.350986:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576447.671651:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:2.0:1788576447.863398:0:2373:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:2.0:1788576447.863445:0:2373:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:2.0:1788576447.863462:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:3.0:1788576447.863590:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576444/real 1788576447] req@ffff9ada4557d180 x1875449616577408/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576469 ref 2 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576447.863616:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d180 x1875449616577408/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576469 ref 2 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576447.869621:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576447.869980:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576447.869988:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616577408:192.168.202.113@tcp:250:kworker.0 00000800:00000100:2.0:1788576447.870225:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:0.0:1788576447.882448:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576444/real 1788576447] req@ffff9ada4557e680 x1875449616577536/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576469 ref 2 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788576447.882490:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616577536/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576469 ref 2 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788576447.882503:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788576447.882508:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788576447.882512:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616577536:192.168.202.113@tcp:38:kworker.0 00000800:00000100:2.0:1788576447.883880:0:2373:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:0.0:1788576447.884073:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576444/real 1788576447] req@ffff9ada4557e300 x1875449616577664/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576469 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788576447.884106:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616577664/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576469 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788576447.884117:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788576447.884123:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788576447.884127:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616577664:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788576448.374999:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576448.695197:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576449.399277:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576449.721321:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576449.911088:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576449.911100:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576449.911104:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576449.911114:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8362 00000100:00080000:0.0:1788576449.911127:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576449.911136:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576449.911172:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616577792/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576449.911211:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576449.911226:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576449.911228:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576449.911232:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8362 00000100:00080000:0.0:1788576449.911236:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576449.911240:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576449.911251:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576449.911263:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616577920/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576449.911273:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576449.911276:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576449.911279:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576449.911283:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8362 00000100:00080000:0.0:1788576449.911286:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576449.911290:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576449.911296:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576449.911306:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616578048/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576449.911387:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616577792:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576449.911449:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576449.911483:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576449/real 1788576449] req@ffff9ada4557e300 x1875449616577792/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576474 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576449.911500:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616577792/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576474 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576449.911508:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576449.911512:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576449.911515:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616577792:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576449.911521:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616577920:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576449.911549:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576449.911561:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576449/real 1788576449] req@ffff9ada4557ed80 x1875449616577920/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576474 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576449.911571:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616577920/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576474 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576449.911578:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576449.911580:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576449.911583:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616577920:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576449.911587:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616578048:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576449.911602:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576449.911610:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576449/real 1788576449] req@ffff9ada4557fb80 x1875449616578048/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576474 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576449.911620:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616578048/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576474 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576449.911628:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576449.911631:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576449.911634:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616578048:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576450.424590:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576450.743842:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576451.447334:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576451.775142:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576452.471304:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576452.795895:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576453.495279:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576453.815135:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576454.519556:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576454.839022:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788576455.035504:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576455.035517:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788576455.035522:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576455.035533:0:91222:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8367 00000100:00080000:2.0:1788576455.035543:0:91222:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576455.035550:0:91222:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788576455.035589:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616578176/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576455.035687:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616578176:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576455.035764:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00080000:2.0:1788576455.035799:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576455.035803:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788576455.035806:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576455.035811:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8367 00000100:00080000:2.0:1788576455.035815:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576455.035819:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576455.035830:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576455.035844:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616578304/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576455.038631:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576455/real 1788576455] req@ffff9ada60627480 x1875449616578176/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576480 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576455.038662:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616578176/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576480 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576455.038674:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576455.038679:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576455.038683:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616578176:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576455.038716:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616578304:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576455.038776:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576455.038794:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576455/real 1788576455] req@ffff9ada60624e00 x1875449616578304/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576480 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576455.038808:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616578304/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576480 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576455.038817:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576455.038821:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576455.038824:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616578304:192.168.202.113@tcp:38:kworker.0 00000100:00080000:2.0:1788576455.039264:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576455.039270:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788576455.039274:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576455.039281:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8367 00000100:00080000:2.0:1788576455.039288:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576455.039295:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576455.039306:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576455.039333:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616578432/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576455.039416:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616578432:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576455.039443:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576455.039455:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576455/real 1788576455] req@ffff9ada60625c00 x1875449616578432/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576480 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576455.039474:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616578432/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576480 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576455.039482:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576455.039485:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576455.039489:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616578432:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576455.543265:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576455.863097:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576456.566994:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576456.887660:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576457.591156:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576457.911378:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576458.617637:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576458.936160:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576459.639028:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576459.961712:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576460.151496:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576460.151507:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576460.151511:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576460.151522:0:91222:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8372 00000100:00080000:0.0:1788576460.151534:0:91222:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576460.151542:0:91222:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576460.151583:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616578560/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576460.151617:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576460.151621:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576460.151623:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576460.151626:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8372 00000100:00080000:0.0:1788576460.151630:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576460.151633:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576460.151643:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576460.151655:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616578688/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576460.151670:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576460.151673:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576460.151675:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576460.151678:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8372 00000100:00080000:0.0:1788576460.151681:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576460.151684:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576460.151689:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576460.151699:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616578816/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576460.151769:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616578560:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576460.151832:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576460.151918:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576460/real 1788576460] req@ffff9ada4557f800 x1875449616578560/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576485 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576460.151937:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616578560/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576485 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576460.151949:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576460.151963:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576460.151968:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616578560:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576460.151976:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616578688:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576460.152011:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576460.152023:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576460/real 1788576460] req@ffff9ada4557dc00 x1875449616578688/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576485 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576460.152035:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616578688/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576485 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576460.152048:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576460.152052:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576460.152055:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616578688:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576460.152060:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616578816:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576460.152078:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576460.152089:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576460/real 1788576460] req@ffff9ada4557d880 x1875449616578816/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576485 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576460.152100:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616578816/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576485 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576460.152106:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576460.152109:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576460.152112:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616578816:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576460.662998:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576460.984843:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576461.687043:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576462.010486:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576462.711260:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576463.031946:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576463.735014:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576464.056971:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576464.761939:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576465.079382:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788576465.271686:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576465.271695:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788576465.271699:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576465.271709:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8377 00000100:00080000:1.0:1788576465.271718:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576465.271724:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788576465.271760:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566e300 x1875449616578944/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788576465.271789:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576465.271792:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788576465.271794:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576465.271797:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8377 00000100:00080000:1.0:1788576465.271801:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576465.271804:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576465.271812:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576465.271821:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f480 x1875449616579072/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788576465.271830:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576465.271833:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788576465.271834:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576465.271837:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8377 00000100:00080000:1.0:1788576465.271840:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576465.271843:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576465.271846:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576465.271880:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ea00 x1875449616579200/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576465.272656:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616578944:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576465.272734:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576465.272772:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576465/real 1788576465] req@ffff9ada4566e300 x1875449616578944/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576490 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576465.272789:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566e300 x1875449616578944/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576490 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576465.272799:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576465.272803:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576465.272807:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616578944:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576465.272814:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616579072:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576465.272836:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576465.272849:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576465/real 1788576465] req@ffff9ada4566f480 x1875449616579072/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576490 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576465.272889:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f480 x1875449616579072/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576490 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576465.272898:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576465.272901:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576465.272904:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616579072:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576465.272910:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616579200:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576465.272930:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576465.272956:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576465/real 1788576465] req@ffff9ada4566ea00 x1875449616579200/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576490 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576465.272968:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ea00 x1875449616579200/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576490 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576465.272975:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576465.272979:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576465.272982:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616579200:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788576465.784124:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576466.105232:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576466.807181:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576467.128990:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576467.831049:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576468.151526:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576468.855911:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576469.175271:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576469.878998:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576470.199621:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576470.391378:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576470.391390:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576470.391395:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576470.391407:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8382 00000100:00080000:0.0:1788576470.391421:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576470.391430:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576470.391475:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616579328/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576470.391563:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616579328:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576470.391627:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00080000:0.0:1788576470.392058:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576470.392063:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576470.392066:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576470.392070:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8382 00000100:00080000:0.0:1788576470.392075:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576470.392080:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576470.392090:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576470.392105:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616579456/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576470.392221:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576470/real 1788576470] req@ffff9ada4557ca80 x1875449616579328/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576495 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576470.392241:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616579328/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576495 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576470.392251:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576470.392254:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576470.392258:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616579328:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576470.392283:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616579456:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576470.392310:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576470.392323:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576470/real 1788576470] req@ffff9ada4557c700 x1875449616579456/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576495 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576470.392335:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616579456/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576495 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576470.392342:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576470.392345:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576470.392348:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616579456:192.168.202.113@tcp:38:kworker.0 00000100:00080000:0.0:1788576470.392439:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576470.392443:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576470.392445:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576470.392450:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8382 00000100:00080000:0.0:1788576470.392454:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576470.392458:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576470.392464:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576470.392478:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ce00 x1875449616579584/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576470.392591:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616579584:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576470.392614:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576470.392797:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576470/real 1788576470] req@ffff9ada4557ce00 x1875449616579584/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576495 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576470.392813:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ce00 x1875449616579584/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576495 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576470.392822:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576470.392825:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576470.392829:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616579584:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576470.903176:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576471.230230:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576471.927419:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576472.247514:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576472.951187:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576473.271585:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576473.977064:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576474.295466:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576474.999278:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576475.321973:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576475.511174:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576475.511184:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576475.511188:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576475.511198:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8388 00000100:00080000:0.0:1788576475.511223:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576475.511230:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576475.511268:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c000 x1875449616579712/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576475.511300:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576475.511305:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576475.511306:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576475.511311:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8388 00000100:00080000:0.0:1788576475.511315:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576475.511318:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576475.511329:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576475.511341:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616579840/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576475.511350:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576475.511353:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576475.511354:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576475.511358:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8388 00000100:00080000:0.0:1788576475.511361:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576475.511364:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576475.511368:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576475.511389:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c380 x1875449616579968/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576475.512658:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616579712:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576475.512732:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576475.512772:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576475/real 1788576475] req@ffff9ada4557c000 x1875449616579712/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576500 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576475.512790:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c000 x1875449616579712/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576500 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576475.512801:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576475.512805:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576475.512808:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616579712:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576475.512816:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616579840:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576475.512839:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576475.512899:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576475/real 1788576475] req@ffff9ada4557f480 x1875449616579840/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576500 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576475.512912:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616579840/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576500 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576475.512924:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576475.512927:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576475.512931:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616579840:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576475.512937:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616579968:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576475.512958:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576475.512969:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576475/real 1788576475] req@ffff9ada4557c380 x1875449616579968/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576500 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576475.512979:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c380 x1875449616579968/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576500 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576475.512986:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576475.512988:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576475.512991:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616579968:192.168.202.113@tcp:38:kworker.0 00000100:00100000:2.0:1788576476.023142:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576476.343942:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788576477.047031:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576477.369238:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576478.071069:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576478.391431:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576479.095002:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576479.415460:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576480.118986:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576480.440459:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788576480.631624:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576480.631645:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788576480.631649:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576480.631661:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8393 00000100:00080000:1.0:1788576480.631672:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576480.631680:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788576480.631720:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d500 x1875449616580096/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576480.631890:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616580096:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576480.631951:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00080000:1.0:1788576480.632000:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576480.632013:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788576480.632015:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576480.632020:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8393 00000100:00080000:1.0:1788576480.632024:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576480.632028:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576480.632037:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576480.632054:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566ca80 x1875449616580224/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576480.640068:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576480/real 1788576480] req@ffff9ada4566d500 x1875449616580096/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576505 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576480.640090:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d500 x1875449616580096/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576505 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576480.640099:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576480.640103:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576480.640106:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616580096:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576480.640133:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616580224:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576480.640184:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576480.640197:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576480/real 1788576480] req@ffff9ada4566ca80 x1875449616580224/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576505 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576480.640205:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566ca80 x1875449616580224/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576505 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576480.640210:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576480.640212:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576480.640214:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616580224:192.168.202.113@tcp:38:kworker.0 00000100:00080000:1.0:1788576480.642513:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576480.642525:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788576480.642529:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576480.642541:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8393 00000100:00080000:1.0:1788576480.642552:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576480.642559:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576480.642572:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576480.642606:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566d180 x1875449616580352/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576480.644366:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616580352:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576480.644450:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576480.644483:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576480/real 1788576480] req@ffff9ada4566d180 x1875449616580352/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576505 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576480.644502:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566d180 x1875449616580352/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576505 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576480.644524:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576480.644528:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576480.644532:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616580352:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788576481.143428:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576481.470559:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576482.166996:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576482.487236:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576483.191892:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576483.511115:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576484.215400:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576484.542010:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788576485.239662:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576485.565173:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1788576485.751332:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576485.751344:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1788576485.751348:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576485.751359:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8398 00000100:00080000:1.0:1788576485.751371:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576485.751379:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:1.0:1788576485.751421:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566f100 x1875449616580480/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788576485.751461:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576485.751465:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1788576485.751467:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576485.751471:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8398 00000100:00080000:1.0:1788576485.751475:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576485.751479:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576485.751490:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576485.751503:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c700 x1875449616580608/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1788576485.751513:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1788576485.751516:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1788576485.751518:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1788576485.751521:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8398 00000100:00080000:1.0:1788576485.751525:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1788576485.751528:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:1.0:1788576485.751533:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1788576485.751543:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4566c380 x1875449616580736/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576485.751650:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616580480:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576485.751714:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576485.751779:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576485/real 1788576485] req@ffff9ada4566f100 x1875449616580480/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576510 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576485.751797:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566f100 x1875449616580480/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576510 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576485.751807:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576485.751811:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576485.751815:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616580480:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576485.751822:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616580608:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576485.751844:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576485.751944:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576485/real 1788576485] req@ffff9ada4566c700 x1875449616580608/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576510 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576485.751957:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c700 x1875449616580608/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576510 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576485.751964:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576485.751967:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576485.751970:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616580608:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576485.751988:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616580736:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576485.752010:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576485.752033:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576485/real 1788576485] req@ffff9ada4566c380 x1875449616580736/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576510 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576485.752044:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4566c380 x1875449616580736/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576510 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576485.752051:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576485.752054:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576485.752057:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4566c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616580736:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788576486.273504:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576486.587019:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788576487.287352:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576487.607229:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788576488.311138:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576488.631658:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788576489.335623:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576489.656211:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788576490.359265:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576490.679653:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576490.871800:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576490.871814:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576490.871819:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576490.871833:0:91222:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8403 00000100:00080000:0.0:1788576490.871849:0:91222:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576490.871889:0:91222:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576490.871964:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c380 x1875449616580864/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576490.871997:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576490.872001:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576490.872003:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576490.872007:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8403 00000100:00080000:0.0:1788576490.872010:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576490.872015:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576490.872027:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576490.872040:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f480 x1875449616580992/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576490.872049:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576490.872057:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576490.872059:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576490.872063:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8403 00000100:00080000:0.0:1788576490.872066:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576490.872070:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576490.872074:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576490.872085:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c000 x1875449616581120/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576490.872234:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616580864:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576490.872315:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576490.872360:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576490/real 1788576490] req@ffff9ada4557c380 x1875449616580864/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576515 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576490.872379:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c380 x1875449616580864/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576515 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576490.872390:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576490.872394:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576490.872398:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616580864:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576490.872405:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616580992:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576490.872426:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576490.872437:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576490/real 1788576490] req@ffff9ada4557f480 x1875449616580992/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576515 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576490.872448:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f480 x1875449616580992/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576515 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576490.872457:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576490.872460:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576490.872464:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616580992:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576490.872470:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616581120:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576490.872487:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576490.872497:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576490/real 1788576490] req@ffff9ada4557c000 x1875449616581120/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576515 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576490.872521:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c000 x1875449616581120/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576515 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576490.872528:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576490.872531:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576490.872534:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616581120:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788576491.383718:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576491.704399:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788576492.407142:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576492.728550:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788576493.431159:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576493.751590:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788576494.455045:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576494.775235:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788576495.479041:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576495.799217:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576495.991328:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576495.991347:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576495.991354:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576495.991370:0:91222:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8408 00000100:00080000:0.0:1788576495.991387:0:91222:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576495.991399:0:91222:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576495.991456:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ce00 x1875449616581248/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576495.991493:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576495.991497:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576495.991499:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576495.991503:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8408 00000100:00080000:0.0:1788576495.991507:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576495.991511:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576495.991523:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576495.991536:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557c700 x1875449616581376/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576495.991545:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576495.991548:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576495.991549:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576495.991553:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8408 00000100:00080000:0.0:1788576495.991556:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576495.991560:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576495.991565:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576495.991575:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ca80 x1875449616581504/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576495.991579:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616581248:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576495.991677:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576495.991919:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576495/real 1788576495] req@ffff9ada4557ce00 x1875449616581248/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576520 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576495.991943:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ce00 x1875449616581248/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576520 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576495.991954:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576495.991958:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576495.991962:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616581248:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576495.991969:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616581376:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576495.991995:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576495.992007:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576495/real 1788576495] req@ffff9ada4557c700 x1875449616581376/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576520 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576495.992018:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557c700 x1875449616581376/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576520 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576495.992027:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576495.992030:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576495.992033:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616581376:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576495.992067:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616581504:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576495.992088:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576495.992099:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576495/real 1788576495] req@ffff9ada4557ca80 x1875449616581504/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576520 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576495.992111:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ca80 x1875449616581504/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576520 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576495.992119:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576495.992122:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576495.992126:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616581504:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788576496.504089:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576496.823187:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788576497.527974:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576497.847456:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788576498.551602:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576498.873833:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788576499.575259:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576499.903358:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788576500.599532:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576500.921608:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576501.111554:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576501.111571:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576501.111578:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576501.111593:0:91222:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8413 00000100:00080000:0.0:1788576501.111606:0:91222:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576501.111616:0:91222:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576501.111670:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d880 x1875449616581632/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576501.111704:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576501.111708:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576501.111710:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576501.111714:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8413 00000100:00080000:0.0:1788576501.111717:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576501.111722:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576501.111744:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576501.111759:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557dc00 x1875449616581760/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576501.111759:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616581632:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576501.111851:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576501.111924:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576501/real 1788576501] req@ffff9ada4557d880 x1875449616581632/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576526 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576501.111948:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d880 x1875449616581632/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576526 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576501.111961:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576501.111967:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576501.111971:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616581632:192.168.202.113@tcp:250:kworker.0 00000100:00080000:0.0:1788576501.112403:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576501.112407:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576501.112409:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576501.112414:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8413 00000100:00080000:0.0:1788576501.112419:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576501.112424:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576501.112430:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576501.112447:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557f800 x1875449616581888/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576501.112511:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616581760:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576501.112557:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576501.112574:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576501/real 1788576501] req@ffff9ada4557dc00 x1875449616581760/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576526 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576501.112588:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557dc00 x1875449616581760/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576526 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576501.112596:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576501.112600:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576501.112604:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616581760:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576501.112610:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616581888:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576501.112632:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576501.112644:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576501/real 1788576501] req@ffff9ada4557f800 x1875449616581888/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576526 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576501.112657:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557f800 x1875449616581888/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576526 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576501.112665:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576501.112668:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576501.112671:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616581888:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788576501.625276:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576501.945102:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788576502.647547:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576502.967228:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788576503.671037:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576503.991879:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788576504.695105:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576505.015201:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788576505.719120:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576506.040444:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788576506.231464:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576506.231478:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788576506.231484:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576506.231496:0:91222:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8418 00000100:00080000:2.0:1788576506.231511:0:91222:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576506.231520:0:91222:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788576506.231573:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616582016/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576506.231609:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576506.231613:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788576506.231615:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576506.231618:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8418 00000100:00080000:2.0:1788576506.231621:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576506.231624:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576506.231644:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576506.231659:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616582144/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576506.231668:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576506.231672:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788576506.231675:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576506.231678:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8418 00000100:00080000:2.0:1788576506.231682:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576506.231686:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576506.231690:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576506.231701:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616582272/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576506.231909:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616582016:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576506.231978:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576506.232022:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576506/real 1788576506] req@ffff9ada60624380 x1875449616582016/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576531 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576506.232039:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616582016/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576531 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576506.232048:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576506.232051:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576506.232054:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616582016:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576506.232059:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616582144:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576506.232075:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576506.232082:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576506/real 1788576506] req@ffff9ada60625880 x1875449616582144/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576531 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576506.232089:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616582144/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576531 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576506.232099:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576506.232101:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576506.232104:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616582144:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576506.232109:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616582272:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576506.232125:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576506.232135:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576506/real 1788576506] req@ffff9ada60624a80 x1875449616582272/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576531 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576506.232146:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616582272/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576531 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576506.232154:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576506.232157:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576506.232160:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616582272:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788576506.743028:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576507.063552:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788576507.767040:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576508.087470:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788576508.791243:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576509.111620:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788576509.818179:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576510.135247:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788576510.839062:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576511.161160:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788576511.352596:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788576511.352607:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788576511.352611:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788576511.352623:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8423 00000100:00080000:3.0:1788576511.352635:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788576511.352642:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788576511.352690:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0380 x1875449616582400/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788576511.352739:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788576511.352742:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788576511.352744:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788576511.352747:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8423 00000100:00080000:3.0:1788576511.352752:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788576511.352755:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788576511.352765:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788576511.352801:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3480 x1875449616582528/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788576511.352810:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788576511.352813:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788576511.352814:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788576511.352817:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8423 00000100:00080000:3.0:1788576511.352820:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788576511.352823:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788576511.352827:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788576511.352838:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0700 x1875449616582656/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788576511.353430:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616582400:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788576511.353650:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616582528:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788576511.353689:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616582656:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788576511.864306:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576512.185169:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788576512.887317:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576513.207934:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788576513.911172:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576514.232318:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:1.0:1788576514.423660:0:2371:0:(lib-socket.c:438:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.202.113:988 00000400:00000100:1.0:1788576514.423721:0:2371:0:(acceptor.c:117:lnet_connect_console_error()) Connection to 192.168.202.113@tcp at host 192.168.202.113 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000800:00000100:1.0:1788576514.423744:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:1.0:1788576514.423804:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000800:00000100:1.0:1788576514.423816:0:2371:0:(socklnd_cb.c:353:ksocknal_txlist_done()) Deleting packet type 1 len 520 0@<0:0>->192.168.202.113@tcp 00000100:00100000:0.0:1788576514.428719:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576511/real 1788576514] req@ffff9ada604e0380 x1875449616582400/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576536 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788576514.428770:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0380 x1875449616582400/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576536 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788576514.428803:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788576514.428809:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788576514.428814:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616582400:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788576514.428825:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576511/real 1788576514] req@ffff9ada604e3480 x1875449616582528/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576536 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788576514.428887:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3480 x1875449616582528/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576536 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788576514.428897:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788576514.428902:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788576514.428906:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616582528:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788576514.428915:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576511/real 1788576514] req@ffff9ada604e0700 x1875449616582656/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576536 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788576514.428930:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0700 x1875449616582656/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576536 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788576514.428938:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788576514.429512:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788576514.429520:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616582656:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788576514.936093:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576515.257898:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576515.959009:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576516.281435:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576516.473364:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576516.473380:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576516.473386:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576516.473571:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8429 00000100:00080000:0.0:1788576516.473586:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576516.473597:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576516.473656:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557fb80 x1875449616582784/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576516.473745:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616582784:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576516.473844:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00080000:0.0:1788576516.473945:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576516.473949:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576516.473952:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576516.473956:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8429 00000100:00080000:0.0:1788576516.473961:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576516.473965:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576516.473978:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576516.474004:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ed80 x1875449616582912/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576516.481650:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576516/real 1788576516] req@ffff9ada4557fb80 x1875449616582784/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576541 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576516.482081:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557fb80 x1875449616582784/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576541 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576516.482097:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576516.482103:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576516.482107:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616582784:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576516.482167:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616582912:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576516.482258:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576516.482287:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576516/real 1788576516] req@ffff9ada4557ed80 x1875449616582912/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576541 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576516.482300:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ed80 x1875449616582912/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576541 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576516.482308:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576516.482311:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576516.482314:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616582912:192.168.202.113@tcp:38:kworker.0 00000100:00080000:0.0:1788576516.482825:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576516.482834:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576516.482837:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576516.482843:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8429 00000100:00080000:0.0:1788576516.482852:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576516.482873:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576516.482889:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576516.482926:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616583040/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576516.486077:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616583040:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576516.486189:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576516.486235:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576516/real 1788576516] req@ffff9ada4557e300 x1875449616583040/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576541 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576516.486257:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616583040/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576541 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576516.486276:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576516.486281:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576516.486286:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616583040:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788576516.983328:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576517.303155:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576518.007354:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576518.327178:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576519.031142:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576519.351163:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576520.055416:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576520.377706:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576521.079215:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576521.400314:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576521.592016:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576521.592043:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576521.592048:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576521.592062:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8434 00000100:00080000:0.0:1788576521.592076:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576521.592086:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576521.592152:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616583168/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576521.592190:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576521.592357:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576521.592360:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576521.592365:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8434 00000100:00080000:0.0:1788576521.592369:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576521.592373:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576521.592384:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576521.592399:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d180 x1875449616583296/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576521.592409:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576521.592411:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576521.592413:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576521.592416:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8434 00000100:00080000:0.0:1788576521.592420:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576521.592424:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576521.592428:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576521.592437:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616583424/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576521.592554:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616583168:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576521.592636:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576521.592686:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576521/real 1788576521] req@ffff9ada4557e680 x1875449616583168/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576546 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576521.592710:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616583168/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576546 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576521.592721:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576521.592726:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576521.592730:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616583168:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576521.592737:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616583296:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576521.592761:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576521.592774:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576521/real 1788576521] req@ffff9ada4557d180 x1875449616583296/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576546 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576521.592784:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d180 x1875449616583296/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576546 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576521.592792:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576521.592795:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576521.592799:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616583296:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576521.592805:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616583424:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576521.592823:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576521.592846:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576521/real 1788576521] req@ffff9ada4557ea00 x1875449616583424/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576546 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576521.592902:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616583424/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576546 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576521.592909:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576521.592912:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576521.592915:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616583424:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788576522.103304:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576522.423340:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576523.127033:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576523.447288:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576524.151221:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576524.474478:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576525.175008:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576525.496723:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576526.199172:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576526.520735:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576526.711069:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576526.711084:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576526.711090:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576526.711103:0:91222:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8439 00000100:00080000:0.0:1788576526.711118:0:91222:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576526.711129:0:91222:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576526.711183:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557df80 x1875449616583552/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576526.711218:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576526.711223:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576526.711224:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576526.711228:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8439 00000100:00080000:0.0:1788576526.711232:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576526.711235:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576526.711248:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576526.711260:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d500 x1875449616583680/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1788576526.711269:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576526.711273:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576526.711275:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576526.711278:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8439 00000100:00080000:0.0:1788576526.711282:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576526.711285:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576526.711290:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576526.711302:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557ea00 x1875449616583808/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576526.711380:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616583552:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576526.711470:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576526.711518:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576526/real 1788576526] req@ffff9ada4557df80 x1875449616583552/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576551 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576526.711539:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557df80 x1875449616583552/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576551 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576526.711551:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576526.711555:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576526.711558:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616583552:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576526.711566:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616583680:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576526.711591:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576526.711603:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576526/real 1788576526] req@ffff9ada4557d500 x1875449616583680/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576551 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576526.711614:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d500 x1875449616583680/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576551 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576526.711622:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576526.711625:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576526.711628:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616583680:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576526.711633:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616583808:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576526.711651:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576526.711661:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576526/real 1788576526] req@ffff9ada4557ea00 x1875449616583808/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576551 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576526.711670:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557ea00 x1875449616583808/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576551 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576526.711678:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576526.711681:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576526.711685:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616583808:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788576527.222999:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576527.543154:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576528.247047:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576528.569270:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576529.271277:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576529.592046:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576530.295294:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576530.617214:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576531.320318:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576531.640452:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788576531.834689:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576531.834703:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788576531.834709:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576531.834722:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8444 00000100:00080000:2.0:1788576531.834738:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576531.834747:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788576531.834808:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616583936/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576531.834844:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576531.834848:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788576531.834850:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1788576531.835083:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616583936:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576531.835181:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576531.835240:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576531/real 1788576531] req@ffff9ada60625500 x1875449616583936/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576556 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576531.835264:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616583936/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576556 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576531.835282:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576531.835287:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576531.835291:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616583936:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788576531.836055:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8444 00000100:00080000:2.0:1788576531.836064:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576531.836073:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576531.836087:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576531.836112:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616584064/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576531.836140:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576531.836145:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788576531.836147:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576531.836151:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8444 00000100:00080000:2.0:1788576531.836155:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576531.836159:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576531.836164:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576531.836175:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627100 x1875449616584192/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576531.837077:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616584064:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576531.837152:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576531.837186:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576531/real 1788576531] req@ffff9ada60626a00 x1875449616584064/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576556 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576531.837205:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616584064/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576556 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576531.837217:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576531.837221:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576531.837225:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616584064:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576531.837232:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616584192:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576531.837254:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576531.837265:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576531/real 1788576531] req@ffff9ada60627100 x1875449616584192/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576556 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576531.837278:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627100 x1875449616584192/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576556 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576531.837290:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576531.837293:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576531.837296:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616584192:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788576532.343402:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576532.668079:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576533.367712:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576533.688807:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788576534.391160:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576534.717632:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788576535.415156:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576535.737762:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576536.440468:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576536.759178:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788576536.953701:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576536.953890:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1788576536.953896:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576536.953919:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8449 00000100:00080000:0.0:1788576536.953934:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576536.953946:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:0.0:1788576536.954004:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557d180 x1875449616584320/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576536.954131:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616584320:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576536.954227:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00080000:0.0:1788576536.954370:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576536.954376:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1788576536.954378:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576536.954383:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8449 00000100:00080000:0.0:1788576536.954387:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576536.954391:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576536.954404:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576536.954419:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e680 x1875449616584448/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576536.954548:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576536/real 1788576536] req@ffff9ada4557d180 x1875449616584320/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576561 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576536.954570:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557d180 x1875449616584320/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576561 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576536.954582:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576536.954587:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576536.954591:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616584320:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576536.954623:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616584448:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576536.954651:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576536.954665:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576536/real 1788576536] req@ffff9ada4557e680 x1875449616584448/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576561 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576536.954676:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e680 x1875449616584448/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576561 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576536.954685:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576536.954688:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576536.954692:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616584448:192.168.202.113@tcp:38:kworker.0 00000100:00080000:0.0:1788576536.955001:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1788576536.955006:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1788576536.955008:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1788576536.955013:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8449 00000100:00080000:0.0:1788576536.955018:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1788576536.955023:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:0.0:1788576536.955030:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1788576536.955045:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada4557e300 x1875449616584576/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576536.955090:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616584576:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576536.955116:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576536.964609:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576536/real 1788576536] req@ffff9ada4557e300 x1875449616584576/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576561 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576536.964650:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada4557e300 x1875449616584576/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576561 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576536.964666:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576536.964672:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576536.964678:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada4557e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616584576:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788576537.466836:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576537.783630:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576538.488422:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576538.807746:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576539.514467:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576539.831186:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576540.535232:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576540.856818:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788576541.561790:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576541.880608:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788576542.075204:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576542.075217:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788576542.075221:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576542.075234:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8454 00000100:00080000:2.0:1788576542.075246:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576542.075254:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788576542.075293:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626680 x1875449616584704/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576542.075324:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576542.075328:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788576542.075330:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576542.075334:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8454 00000100:00080000:2.0:1788576542.075338:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576542.075342:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576542.075354:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576542.075367:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624700 x1875449616584832/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576542.075377:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576542.075380:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788576542.075381:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576542.075385:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8454 00000100:00080000:2.0:1788576542.075388:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576542.075392:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576542.075397:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576542.075408:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627100 x1875449616584960/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576542.075482:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616584704:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576542.075548:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576542.075583:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576542/real 1788576542] req@ffff9ada60626680 x1875449616584704/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576567 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576542.075601:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626680 x1875449616584704/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576567 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576542.075610:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576542.075614:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576542.075618:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616584704:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576542.075625:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616584832:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576542.075646:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576542.075658:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576542/real 1788576542] req@ffff9ada60624700 x1875449616584832/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576567 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576542.075669:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624700 x1875449616584832/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576567 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576542.075677:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576542.075680:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576542.075684:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616584832:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576542.075689:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616584960:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576542.075708:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576542.075719:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576542/real 1788576542] req@ffff9ada60627100 x1875449616584960/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576567 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576542.075729:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627100 x1875449616584960/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576567 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576542.075736:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576542.075740:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576542.075743:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616584960:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788576542.583542:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576542.903226:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576543.607422:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576543.927166:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576544.631237:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576544.954501:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788576545.661246:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576545.977430:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788576546.679141:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576546.999663:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788576547.192874:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576547.192885:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788576547.192889:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576547.192899:0:91222:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8459 00000100:00080000:2.0:1788576547.192910:0:91222:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576547.192918:0:91222:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788576547.192971:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60626a00 x1875449616585088/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576547.193011:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576547.193014:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788576547.193016:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576547.193019:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8459 00000100:00080000:2.0:1788576547.193023:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576547.193027:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576547.193037:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576547.193049:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625500 x1875449616585216/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576547.193058:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576547.193061:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788576547.193063:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576547.193067:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8459 00000100:00080000:2.0:1788576547.193070:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576547.193073:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576547.193077:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576547.193099:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624a80 x1875449616585344/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576547.194898:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616585088:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576547.195013:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576547.195062:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576547/real 1788576547] req@ffff9ada60626a00 x1875449616585088/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576572 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576547.195082:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60626a00 x1875449616585088/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576572 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576547.195103:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576547.195107:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576547.195111:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60626a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616585088:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576547.195127:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616585216:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576547.195150:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576547.195162:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576547/real 1788576547] req@ffff9ada60625500 x1875449616585216/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576572 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576547.195175:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625500 x1875449616585216/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576572 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576547.195195:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576547.195199:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576547.195201:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616585216:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576547.195208:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616585344:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576547.195227:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576547.195238:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576547/real 1788576547] req@ffff9ada60624a80 x1875449616585344/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576572 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576547.195250:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624a80 x1875449616585344/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576572 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576547.195264:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576547.195268:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576547.195271:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616585344:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788576547.708095:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576548.023139:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788576548.727050:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576549.047193:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788576549.751336:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576550.071347:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788576550.775637:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576551.095183:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576551.800728:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576552.121417:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788576552.314438:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576552.314454:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788576552.314460:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576552.314475:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8464 00000100:00080000:2.0:1788576552.314489:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576552.314497:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788576552.314553:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625880 x1875449616585472/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576552.314592:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576552.314597:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788576552.314599:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576552.314603:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8464 00000100:00080000:2.0:1788576552.314608:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576552.314612:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576552.314623:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576552.314637:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624380 x1875449616585600/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576552.314647:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576552.314651:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788576552.314653:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576552.314656:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8464 00000100:00080000:2.0:1788576552.314660:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576552.314663:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576552.314667:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576552.314677:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625c00 x1875449616585728/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1788576552.314768:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616585472:192.168.202.113@tcp:250:kworker.0 00000800:00000100:3.0:1788576552.314906:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576552.314950:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576552/real 1788576552] req@ffff9ada60625880 x1875449616585472/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576577 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576552.314972:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625880 x1875449616585472/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576577 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576552.314997:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576552.315002:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576552.315006:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616585472:192.168.202.113@tcp:250:kworker.0 00000100:00100000:3.0:1788576552.315014:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616585600:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576552.315041:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576552.315054:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576552/real 1788576552] req@ffff9ada60624380 x1875449616585600/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576577 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576552.315066:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624380 x1875449616585600/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576577 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576552.315075:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576552.315079:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576552.315083:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616585600:192.168.202.113@tcp:38:kworker.0 00000100:00100000:3.0:1788576552.315089:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616585728:192.168.202.113@tcp:38:kworker.0 00000800:00000100:3.0:1788576552.315351:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:3.0:1788576552.315366:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576552/real 1788576552] req@ffff9ada60625c00 x1875449616585728/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576577 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788576552.315378:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625c00 x1875449616585728/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576577 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788576552.315386:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1788576552.315389:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:3.0:1788576552.315393:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616585728:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788576552.827382:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576553.150585:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576553.847047:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576554.174137:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576554.871427:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576555.191779:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576555.896912:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576556.216793:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576556.922413:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576557.239226:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788576557.431766:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576557.431780:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788576557.431785:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576557.431798:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8469 00000100:00080000:2.0:1788576557.431813:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576557.431822:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788576557.435117:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60624e00 x1875449616585856/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788576557.435259:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616585856:192.168.202.113@tcp:250:kworker.0 00000800:00000100:0.0:1788576557.435356:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00080000:3.0:1788576557.435728:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788576557.435734:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788576557.435737:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788576557.435744:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8469 00000100:00080000:3.0:1788576557.435751:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788576557.435759:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788576557.435774:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788576557.435797:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1180 x1875449616585984/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788576557.443313:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576557/real 1788576557] req@ffff9ada60624e00 x1875449616585856/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576582 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788576557.443349:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60624e00 x1875449616585856/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576582 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788576557.443364:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788576557.443370:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788576557.443374:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60624e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616585856:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788576557.444261:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616585984:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788576557.444346:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788576557.444371:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576557/real 1788576557] req@ffff9ada604e1180 x1875449616585984/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576582 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788576557.444560:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1180 x1875449616585984/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576582 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788576557.444571:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788576557.444575:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788576557.444579:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616585984:192.168.202.113@tcp:38:kworker.0 00000100:00080000:3.0:1788576557.447806:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788576557.447821:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788576557.447826:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788576557.447839:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8469 00000100:00080000:3.0:1788576557.447868:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788576557.447884:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788576557.447901:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788576557.447947:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e3800 x1875449616586112/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788576557.448486:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616586112:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788576557.448581:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788576557.448616:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576557/real 1788576557] req@ffff9ada604e3800 x1875449616586112/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576582 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788576557.448749:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e3800 x1875449616586112/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576582 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788576557.448761:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788576557.448765:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788576557.448769:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616586112:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788576557.943267:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576558.263472:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576558.967034:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576559.294130:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576559.991490:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576560.311225:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576561.017376:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576561.336195:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576562.039032:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576562.360927:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1788576562.551007:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788576562.551019:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1788576562.551024:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788576562.551037:0:91222:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8475 00000100:00080000:3.0:1788576562.551051:0:91222:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788576562.551061:0:91222:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:3.0:1788576562.551196:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e1880 x1875449616586240/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788576562.551238:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788576562.551242:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1788576562.551244:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788576562.551249:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8475 00000100:00080000:3.0:1788576562.551254:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788576562.551258:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788576562.551269:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788576562.551284:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e2d80 x1875449616586368/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1788576562.551293:0:91222:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1788576562.551296:0:91222:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1788576562.551298:0:91222:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1788576562.551302:0:91222:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8475 00000100:00080000:3.0:1788576562.551305:0:91222:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1788576562.551308:0:91222:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:3.0:1788576562.551312:0:91222:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1788576562.551322:0:91222:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada604e0a80 x1875449616586496/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788576562.553057:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616586240:192.168.202.113@tcp:250:kworker.0 00000800:00000100:0.0:1788576562.553263:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788576562.553316:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576562/real 1788576562] req@ffff9ada604e1880 x1875449616586240/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576587 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788576562.553337:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e1880 x1875449616586240/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576587 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788576562.553350:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788576562.553365:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788576562.553370:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616586240:192.168.202.113@tcp:250:kworker.0 00000100:00100000:0.0:1788576562.553378:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616586368:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788576562.553404:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788576562.553418:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576562/real 1788576562] req@ffff9ada604e2d80 x1875449616586368/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576587 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788576562.553429:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e2d80 x1875449616586368/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576587 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788576562.553438:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788576562.553442:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788576562.553445:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616586368:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788576562.553452:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616586496:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788576562.553474:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788576562.553485:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576562/real 1788576562] req@ffff9ada604e0a80 x1875449616586496/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576587 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788576562.553496:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada604e0a80 x1875449616586496/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576587 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788576562.553504:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788576562.553507:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788576562.553509:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada604e0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616586496:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788576563.062999:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576563.383189:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576564.086999:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576564.408882:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576565.111019:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:3.0:1788576565.431222:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576566.135748:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576566.457041:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576567.158982:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576567.485969:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1788576567.671606:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 90fa6266-5a39-4f6d-b3cf-5043b0cc0e22->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576567.671620:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1788576567.671625:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576567.671638:0:85951:0:(import.c:545:import_select_connection()) MGC192.168.202.113@tcp: connect to NID 192.168.202.113@tcp last attempt 8480 00000100:00080000:2.0:1788576567.671653:0:85951:0:(import.c:630:import_select_connection()) MGC192.168.202.113@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576567.671662:0:85951:0:(import.c:665:import_select_connection()) MGC192.168.202.113@tcp: import ffff9ada58a62800 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000100:00100000:2.0:1788576567.671721:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60627480 x1875449616586624/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576567.671765:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576567.671770:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1788576567.671772:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576567.671776:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8480 00000100:00080000:2.0:1788576567.671780:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1788576567.671928:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616586624:192.168.202.113@tcp:250:kworker.0 00000800:00000100:0.0:1788576567.672015:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788576567.672078:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576567/real 1788576567] req@ffff9ada60627480 x1875449616586624/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576592 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788576567.672099:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60627480 x1875449616586624/t0(0) o250->MGC192.168.202.113@tcp@192.168.202.113@tcp:26/25 lens 520/544 e 0 to 1 dl 1788576592 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788576567.672121:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58a62800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788576567.672126:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of MGS on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788576567.672130:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60627480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:90fa6266-5a39-4f6d-b3cf-5043b0cc0e22:2382:1875449616586624:192.168.202.113@tcp:250:kworker.0 00000100:00080000:2.0:1788576567.672607:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0000-mdc-ffff9ada58b0a000: import ffff9ada51140000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576567.672621:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576567.672641:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625f80 x1875449616586752/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1788576567.672662:0:85951:0:(pinger.c:229:ptlrpc_pinger_process_import()) 35eb2d60-2de0-4b00-9815-d5153edd0057->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1788576567.672666:0:85951:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1788576567.672669:0:85951:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1788576567.672673:0:85951:0:(import.c:545:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: connect to NID 192.168.202.113@tcp last attempt 8480 00000100:00080000:2.0:1788576567.672678:0:85951:0:(import.c:630:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1788576567.672682:0:85951:0:(import.c:665:import_select_connection()) lustre-MDT0001-mdc-ffff9ada58b0a000: import ffff9ada58b08000 using connection 192.168.202.113@tcp/192.168.202.113@tcp 00000008:00100000:2.0:1788576567.672687:0:85951:0:(osc_request.c:3830:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34683394 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1788576567.672697:0:85951:0:(import.c:870:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9ada60625180 x1875449616586880/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1788576567.672768:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616586752:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788576567.673078:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788576567.673097:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576567/real 1788576567] req@ffff9ada60625f80 x1875449616586752/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576592 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788576567.679288:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625f80 x1875449616586752/t0(0) o38->lustre-MDT0000-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576592 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788576567.679328:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada51140000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788576567.679335:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788576567.679343:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616586752:192.168.202.113@tcp:38:kworker.0 00000100:00100000:0.0:1788576567.679361:0:2382:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616586880:192.168.202.113@tcp:38:kworker.0 00000800:00000100:0.0:1788576567.679470:0:2382:0:(socklnd_cb.c:906:ksocknal_launch_packet()) No usable routes to 12345-192.168.202.113@tcp 00000100:00100000:0.0:1788576567.679512:0:2382:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1788576567/real 1788576567] req@ffff9ada60625180 x1875449616586880/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576592 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788576567.679531:0:2382:0:(client.c:2535:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff9ada60625180 x1875449616586880/t0(0) o38->lustre-MDT0001-mdc-ffff9ada58b0a000@192.168.202.113@tcp:12/10 lens 520/544 e 0 to 1 dl 1788576592 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1788576567.679540:0:2382:0:(import.c:65:import_set_state_nolock()) ffff9ada58b08000 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1788576567.679544:0:2382:0:(import.c:1555:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.202.113@tcp failed (-110) 00000100:00100000:0.0:1788576567.679547:0:2382:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff9ada60625180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:35eb2d60-2de0-4b00-9815-d5153edd0057:2382:1875449616586880:192.168.202.113@tcp:38:kworker.0 00000100:00100000:1.0:1788576568.184526:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576568.503135:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576569.207003:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576569.527893:0:15204:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788576570.231110:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576570.552565:0:33423:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788576571.257402:0:93093:0:(client.c:2716:ptlrpc_set_wait()) set ffff9ada507a2cc0 going to sleep for 0 seconds 00010000:00010000:1.0:1788576571.575872:0:15205:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) Debug log: 84533 lines, 84533 kept, 0 dropped, 0 bad.